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Abstract 

The  use  of  computer  simulation  in  the  development  of  cooperatively  controlled 
unmanned  combat  aerial  vehicles  (UCAV)  for  autonomous  wide  area  search  and 
engagement  applications  is  addressed.  Computer  simulation  is  an  essential  tool  to 
analyze  cooperative  control  algorithms  designed  to  optimally  employ  multiple  UCAVs  in 
wide  area  search  and  engagement.  To  be  representative  of  real  world  mission  conditions 
a  simulation  must  be  able  to  accurately  duplicate  the  performance  of  automatic  target 
recognition  (ATR)  algorithms  that  will  be  used  to  discriminate  between  targets  and  non¬ 
targets  in  the  battle  space.  The  objective  of  this  research  is  to  demonstrate  a  method  to 
validate  a  simulation’s  ATR  model  for  future  use  in  the  evaluation  of  cooperative  control 
schemes.  This  is  accomplished  by  comparing  the  results  from  multiple  simulations  of 
academically  contrived  wide  area  search  and  engagement  scenarios  to  closed  form 
analytic  solutions  derived  for  the  same  scenarios. 
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WIDE  AREA  SEARCH  AND  ENGAGEMENT 
SIMULATION  VALIDATION 


I.  Introduction 


Background 

The  Department  of  Defense  (DoD)  is  interested  in  the  development  of  low  cost 
autonomous  unmanned  combat  aerial  vehicles  (UCAV’s)  for  use  in  wide  area  search  and 
engagement  applications.  Warfighters  envision  a  weapon  system  that  can  locate,  identify 
and  attack  select  targets  in  a  given  battle  space  with  minimal  human  direction. 
Operational  scenarios  for  this  class  of  UCAV  could  include  Theater  Missile  Defense 
(TMD),  Suppression  of  Enemy  Air  Defense  (SEAD),  Persistent  Area  Denial  (PAD), 
Urban  Battlefield  Surveillance  (UBS)  and  any  other  application  that  involves  searching 
defined  geographic  areas  for  specific  target  types. 

One  example  of  such  a  system  is  the  Low  Cost  Autonomous  Attack  System 
(LOCAAS)  shown  in  Figure  1 .  The  LOCAAS  is  a  concept  demonstration  of  an 
autonomous  wide  area  search  and  engagement  system  developed  jointly  by  the  Air  Force 
Research  Lab  Munitions  Directorate  (AFRL/MN),  Eglin  AFB,  Florida  and  Lockheed 
Martin  Missiles  and  Fire  Control,  Orlando,  Florida.  It  is  a  one -time-use  wide  area  search 
munition  (WASM)  designed  to  autonomously  locate  and  attack  ground  mobile  targets 
with  a  multi-mode  explosively  formed  penetrator  warhead.  The  LOCAAS  uses  a  Laser 
Radar  (LADAR)  seeker  coupled  with  automatic  target  recognition  (ATR)  algorithms  and 
an  autonomous  guidance  and  control  system  to  detect,  identify  and  engage  enemy  targets. 
It  can  be  deployed  from  an  aircraft  or  ground  based  launch  system  in  groups  or 
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individually  with  an  expected  endurance  in  the  battle  space  of  up  to  30  minutes. 
Depending  on  mission  objectives,  the  LOCAAS  can  be  programmed  to  attack  the  first 
target  it  encounters  or  wait  for  a  better  target  to  come  along  before  its  endurance  runs  out. 
[14] 


(Lockheed  Martin  photo) 

Figure  1.  Low  Cost  Autonomous  Attack  System  (LOCAAS) 

The  LOCAAS  ATR  algorithms  identify  targets  by  comparing  the  LADAR  seeker 
image  of  an  encountered  object  to  a  preprogrammed  library  of  target  models.  If  the 
image  is  a  quantified  match  to  a  target  model  the  ATR  will  declare  the  object  to  be  a 
target  that  can  be  attacked.  Otherwise,  the  object  is  declared  a  false  target  that  cannot  be 
attacked.  Although  this  is  an  oversimplification  of  the  LOCAAS  ATR,  this  description 
encapsulates  the  rudimentary  ATR  process  that  will  be  used  in  most  autonomous  wide 
area  search  and  engagement  systems. 
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The  reliability  of  an  ATR  system  to  correctly  classify  encountered  objects  is  of 
major  concern.  There  always  exists  some  probability  that  an  ATR  system  will  incorrectly 
classify  an  object.  The  two  types  of  possible  errors  are  a  false  positive  error  where  a 
false  target  is  declared  a  target,  and  a  false  negative  error  where  a  target  is  declared  a 
false  target.  False  positive  errors  will  lead  to  false  target  attacks  (i.e.  collateral  damage) 
and  false  negative  errors  infer  lost  opportunities  for  target  attacks.  The  expected 
frequency  of  these  errors  must  drive  the  development  and  operational  use  of  any 
autonomous  wide  area  search  and  engagement  system.  The  goal  of  a  wide  area  search 
and  engagement  system  should  be  to  maximize  the  number  of  target  kills  possible  while 
limiting  the  number  of  false  target  attacks. 

Previous  Research 

The  Air  Force  Institute  of  Technology  (AFIT)  has  produced  a  great  deal  of 
research  on  the  wide  area  search  and  engagement  problem  over  the  past  decade.  The 
foundation  of  this  body  of  research,  in  particular  Jacques  and  Pachter  [7],  and 
subsequently  Decker  [1]  and  Kish  [8],  has  dealt  primarily  with  developing  closed  form 
analytic  expressions  for  the  expected  outcomes  (i.e.  target  and  false  target  attacks)  of 
various  wide  area  search  and  engagement  scenarios.  Some  of  this  analytic  work, 
principally  [7],  has  been  applied  in  the  Masters  Theses  of  Gillen  [4],  Dunkel  [3], 
Gozaydin  [5]  and  Park  [10]  to  appraise  the  benefits  of  cooperative  behavior  among 
multiple  search  agents.  The  common  thread  in  their  research  was  the  use  of  computer 
simulations  to  analyze  various  cooperative  behavior  schemes.  The  Masters  Thesis  of 
Schulz  [13]  focused  on  validating  the  simulation  tools  used  to  evaluate  wide  area  search 
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and  engagement  cooperative  control  algorithms.  Cumulatively,  this  research  body 
represents  an  advance  in  the  underlying  DoD  goal  of  developing  autonomous  wide  area 
search  and  engagement  systems. 

Jacques  and  Pachter  [7]  were  the  first  at  AFIT  to  derive  analytic  expressions  to 
calculate  the  probabilities  of  target  attack  (Pta)  and  false  target  attack  (Pita)  for  multiple 
wide  area  search  and  engagement  scenarios.  They  considered  a  single  UCAV,  armed 
with  a  single  warhead  performing  an  exhaustive  and  non-duplicative  search  of  a  battle 
space  containing  multiple  targets  and  false  targets.  Their  work  examined  six  wide  area 
search  and  engagement  scenarios  defined  by  different  a  priori  distributions  of  targets  and 
false  targets.  These  scenarios,  listed  below,  have  been  the  basis  for  most  of  the  AFIT 
research  described  in  this  section. 

Scenario  1 :  A  rectangular  battle  space  region  containing  a  single  target  with  a 
uniform  distribution  and  a  Poisson  field  of  false  targets. 

Scenario  2:  Same  battle  space  as  Scenario  1  containing  a  Poisson  field  of  targets 
and  a  Poisson  field  of  false  targets. 

Scenario  3:  Same  battle  space  as  Scenario  1  containing  N  targets  with  a  uniform 
distribution  and  a  Poisson  field  of  false  targets. 

Scenario  4:  Same  battle  space  as  Scenario  1  containing  N  targets  and  M  false 
targets  with  uniform  distributions. 

Scenario  5:  A  circular  battle  space  containing  N  targets  with  a  circular  normal 
distribution  and  a  Poisson  field  of  false  targets. 
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Scenario  6:  Same  battle  space  as  Scenario  5  containing  N  targets  and  M  false 
targets  with  a  circular  nonnal  distribution. 

Decker  [1]  extended  the  work  of  Jacques  and  Pachter  [7]  by  deriving  analytic 
expressions  for  a  multiple  warhead  capable  UCAV  for  the  six  scenarios  described  above. 
He  proposed  methods  for  using  his  analytical  expressions  to  evaluate  cooperation 
schemes  and  rules  of  engagement  for  multiple  search  agents.  Decker  also  suggested 
ways  to  formulate  a  control  problem  to  optimally  employ  a  wide  area  search  and 
engagement  UCAV  using  a  selectable  sensor  threshold  as  the  control  parameter.  He 
proposed  that  by  manipulating  sensor  parameters  as  modeled  by  a  Receiver  Operating 
Characteristic  (ROC)  curve  the  expected  Pta  for  a  system  could  be  improved.  A  ROC 
curve  is  an  experimentally  determined  relationship  between  sensor  ATR  probabilities  for 
correct  target  declarations  to  false  positive  errors. 

Kish  [8]  further  developed  a  ROC  curve  based  control  problem  formulation  to 
maximize  the  Pta  subject  to  a  constrained  Pita-  He  developed  a  generalized  dynamic 
optimization  mathematical  framework  for  all  of  the  scenarios  developed  by  Jacques  and 
Pachter.  His  analytic  expressions  produce  time  varying  optimal  schedules  of  control 
input  parameters  for  sensor  gain  settings  and  area  coverage  rates  (i.e.  UCAV  velocity  and 
altitude)  during  a  mission. 

Other  research  has  applied  the  work  of  Jacques  and  Pachter  [7]  to  evaluate 
cooperative  behavior  of  multiple  autonomous  UCAVs  in  wide  area  search  and 
engagement.  In  [6],  Jacques  provided  some  limited  analysis  of  cooperative  engagement 
of  multiple  single  warhead  UCAVs  searching  a  common  battle  space.  Jacques  discussed 
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two  classes  of  cooperative  behavior:  cooperative  engagement  and  cooperative 
classification.  Cooperative  engagement  is  when  one  or  more  UCAVs  attack  a  target  that 
was  detected  and  classified  by  another.  Cooperative  classification  involves  two  or  more 
UCAVs  perfonning  a  sensor  sweep  of  a  target  in  order  to  classify  it.  Jacques  also 
discussed  the  importance  of  false  target  attack  rates  and  the  distribution  of  targets  as 
critical  factors  to  the  wide  area  search  and  engagement  problem.  For  scenarios  with  high 
densities  of  false  targets,  cooperative  engagement  was  potentially  detrimental  because  the 
possibility  of  a  false  target  attack  becomes  more  likely  before  a  real  target  is  even 
encountered. 

Some  of  Jacques’  students  expanded  on  this  research.  Dunkel  [3]  and  Gozaydin 
[5]  showed  that  cooperative  behavior  can  decrease  the  occurrence  of  targets  attacks  and 
increase  false  target  attacks.  Their  research  considered  cooperative  and  non-cooperative 
scenarios  of  multiple  single  warhead  UCAVs  searching  a  common  battle  space.  Dunkel 
found  that  cooperative  engagement  was  likely  to  result  in  a  higher  false  target  attack  rate 
than  non-cooperative  behavior.  He  concluded  that  this  was  because  there  were  typically 
several  UCAVs  ready  to  attack  any  false  targets  encountered  that  were  classified  as  a 
targets.  Dunkel  did  note  that  cooperative  classification  offered  improved  ATR 
declaration  accuracy.  Both  Dunkel  and  Gozaydin  found  that  the  quantity  of  target  attacks 
decreased  with  cooperative  classification  because  extra  care  and  time  was  spent 
classifying  fewer  encountered  targets.  The  endurance  of  a  group  of  UCAVs  would 
simply  run  out  before  many  targets  could  be  classified  and  attacked. 
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Dunkel  and  Gozaydin  used  computer  simulations  to  generate  their  results.  The 
simulation  they  used  was  the  MultiUAV  [11]  simulation  environment  developed  by  the 
Air  Force  Research  Lab,  Air  Vehicles  Directorate,  Control  Sciences  Division 
(AFRL/VAC),  Wright-Patterson  AFB,  Ohio.  The  simulation  was  written  specifically  to 
analyze  cooperative  control  algorithms  for  autonomous  wide  area  search  and  engagement 
applications.  In  early  versions  of  MultiUAV,  the  probability  for  correctly  classifying 
targets  was  dependent  only  on  the  number  of  sensor  sweeps  executed  on  a  target  by  one 
or  more  simulated  UCAVs.  Correct  classification  was  guaranteed  provided  the 
simulation  time  did  not  run  out  before  a  sufficient  number  of  sensor  sweeps  could  be 
performed.  This  did  not  adequately  duplicate  a  realistic  sensor/ATR  system  process 
performance  because  no  consideration  was  made  for  target  declaration  errors. 

In  [3],  Dunkel  modified  MultiUAV  to  allow  false  positive  and  false  negative 
errors  to  occur  in  the  simulation  ATR  process.  His  results  compared  favorably  with  the 
analytic  work  of  Jacques  and  Pachter.  Schulz  took  this  one  step  farther  and  validated  the 
MultiUAV  simulation  results  against  the  analytical  solutions  for  all  six  scenarios. 
Schulz’s  work  was  limited  to  a  UCAV  armed  with  a  single  warhead  and  constant  ATR 
performance  parameters.  This  research  effort  represents  a  similar  validation  of  the 
MultiUAV  simulation  as  the  work  of  Schulz  [13]  except  that  it  covers  modifications  to 
the  MultiUAV  simulation  that  allow  for  multiple  warheads  per  UCAV  as  well  as  time 
varying  ATR  performance  control  parameters. 
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Research  Objective 

The  primary  goal  of  this  research  is  to  demonstrate  a  method  to  validate  a  generic 
simulation  environment,  such  as  MultiUAV,  for  use  in  the  evaluation  of  cooperatively 
controlled  autonomous  UAVs  in  wide  area  search  and  engagement  applications.  More 
specific  objectives  are: 

1 .  Empirically  characterize  the  occurrence  of  target  and  false  target  attacks  in 
MultiUAV  given  a  single  search  agent  with  multiple  warheads  and 
compare  these  results  to  analytically  derived  solutions. 

2.  Evaluate  the  ability  of  MultiUAV  to  incorporate  time  varying  parameters 
to  optimize  the  simulated  ATR  performance  for  wide  area  search  and 
engagement  research  applications. 

3.  Identify  deficiencies  (if  any)  in  MultiUAV  that  are  uncovered  during  this 
research  effort. 

Methodology  and  Scope 

This  work  presents  a  method  to  validate  the  ability  of  a  simulation  environment 
such  as  MultiUAV  to  realistically  duplicate  the  performance  of  ATR  algorithms  used  to 
discriminate  between  targets  and  false  targets  in  wide  area  search  and  engagement 
applications.  This  was  accomplished  by  comparing  the  results  from  MultiUAV 
simulation  runs  of  Scenario  1  and  Scenario  2  to  the  analytic  framework  developed  by 
Decker  and  Kish.  Scenario  2  was  used  to  evaluate  the  impact  of  multiple  warheads  on 
the  perfonnance  of  MultiUAV  (Objective  1).  Scenario  1  was  used  to  evaluate  the  ability 


8 


to  apply  time  varying  parameters  to  optimally  modify  the  performance  of  the  ATR  model 
in  MultiUAV  (Objective  2).  The  following  perfonnance  parameters  were  examined: 

•  Probability  of  target  attack,  PTA 

•  Probability  of  false  target  attack,  Pfta 

•  Expected  number  of  target  attacks,  ETa 

•  Expected  number  of  false  target  attacks,  EFta 

This  research  does  not  consider  the  case  of  multiple  UCAVs  in  the  battle  space  or 
evaluate  the  performance  of  any  specific  UCAV  design  concepts  or  wide  area  search  and 
engagement  applications.  Instead,  this  work  provides  a  baseline  validation  and  analysis 
of  the  MultiUAV  simulation  environment  for  future  use  in  wide  area  search  and 
engagement  research  applications  (Objective  3). 

Overview  of  Thesis 

The  following  chapter,  Chapter  2,  further  develops  the  wide  area  search  and 
engagement  problem  setup  including  definitions  for  the  battle  space  region  and  the 
distribution  of  targets  and  false  targets  in  the  battle  space.  The  fundamental  analytic 
theory  for  calculating  Pta  and  Pfta  as  well  as  the  formulation  of  the  basic  control 
problem  are  also  provided.  Chapter  3  gives  a  general  description  of  the  MultiUAV 
simulation  environment  and  the  required  setup  and  modifications  to  MultiUAV  specific 
to  this  research.  Chapter  4  will  present  the  empirical  results  from  various  MultiUAV 
simulations  that  were  run  to  accomplish  the  research  objectives  and  how  these  results 
compare  to  the  analytic  theory  from  Chapter  2.  Finally,  Chapter  5  provides  the 
conclusions  of  this  work  and  recommendations  for  further  research. 
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II.  Analytic  Background  for  Wide  Area  Search  and  Engagement 


Chapter  Overview 

Potential  uses  for  wide  area  search  and  engagement  systems  could  include 
Theater  Missile  Defense  (TMD),  Suppression  of  Enemy  Air  Defense  (SEAD),  Persistent 
Area  Denial  (PAD),  Urban  Battlefield  Surveillance  (UBS)  and  many  other  applications 
that  involve  searching  defined  geographic  areas  for  specific  targets.  These  mission  types 
can  differ  greatly  in  terms  of  the  size  of  the  battle  space  and  density  of  targets  and  false 
targets  in  that  battle  space.  For  example,  a  TMD  scenario  may  consist  of  a  large  search 
area  covering  hundreds  of  square  miles  containing  only  a  few  targets  and  false  targets.  In 
contrast,  a  UBS  scenario  might  be  restricted  to  a  few  city  blocks  with  one  or  two 
potential  targets  mixed  in  a  high-density  field  of  false  targets.  To  construct  a  computer 
simulation  that  models  these  types  of  scenarios  the  battle  space  and  target  densities  need 
to  be  well  defined.  Validation  of  simulation  results  may  then  be  accomplished  through 
direct  comparison  with  analytic  solutions  developed  for  each  scenario. 

This  chapter  describes  the  generic  battle  space  and  distributions  of  targets  and 
false  targets  used  to  model  wide  area  search  and  engagement  applications.  In  addition, 
the  chapter  defines  the  perfonnance  of  an  ATR  algorithm  in  terms  of  the  probability  of 
correct  target  classifications.  From  this,  a  brief  overview  is  given  of  the  fundamental 
analytic  theory  for  calculating  Pja  and  Pfta  for  the  academic  Scenarios  1  and  2.  Finally 
the  basic  control  problem  formulation  is  developed  by  modeling  ATR  performance  in 
tenns  of  a  ROC  curve  for  the  single  warhead  UCAV  case  of  Scenario  1. 
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Battle  Space  Description 

Jacques’  and  Pachter’s  [7]  six  wide  area  search  and  engagement  scenarios  utilized 
either  a  rectangular  (Scenarios  1-4)  or  circular  (Scenarios  5  and  6)  battle  space  region. 
This  work  is  confined  to  Scenarios  1  and  2;  therefore,  only  a  rectangular  battle  space  as 
shown  in  Figure  2  is  considered. 


A=WVt  AA  =  WVdt 


A=WVTS 


Figure  2.  Battle  Space  Diagram 

From  Figure  2,  the  UCAV  searches  the  regions  As  with  area  As  at  a  constant 
altitude  and  velocity  V  over  some  total  time  interval  Ts.  The  area  that  has  been  searched 
up  to  time  t,  0  <  t  <Ts,  is  denoted  as  A.  The  sensor  footprint  of  the  UCAV  can  be 
represented  by  an  incremental  area  AA  defined  by  velocity  V,  the  sensor  swath  width  W, 
and  some  time  step  dt.  It  is  convenient  to  normalize  time  t  and  the  area  searched  A  to  the 
variable  x,  0  <  x  <  1 ,  where 
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(1) 


t 


and 

A  =  Asx.  (2) 


Distribution  of  Targets 

For  Scenario  1,  region  As  contains  a  single  target  with  a  uniform  distribution  and 
a  Poisson  field  of  false  targets.  As  a  property  of  the  uniform  distribution  the  location  of 
the  single  target  in  As  can  be  said  to  be  completely  random.  The  probability  that  a  search 
agent  encounters  the  target  in  any  arbitrary  sub-region  of  As  defined  as  having  area  A  is 

/’(Target  in  A)  =  — .  (3) 

A 

The  Poisson  field  of  false  targets  in  As  is  characterized  by  a  probability  density 
parameter  a  which  is  the  expected  number  of  false  target  encounters  per  unit  area.  The 
Poisson  probability  parameter  Aft,  or  expected  quantity  of  false  targets  in  area  A,  is  then 

Aft  =  a  A ,  (4) 

and  the  probability  of  encountering  exactly  k  false  targets  in  area  A  is  defined  as 

2  k 

P[FT  =  k)  =  e~XfT  — k  =  0,1,2...  (5) 

For  Scenario  2,  As  contains  a  Poisson  field  of  targets  and  a  Poisson  field  of  false 
targets.  The  Poisson  field  of  false  targets  remains  the  same  as  in  Scenario  1.  The 
Poisson  field  of  targets  is  characterized  by  a  probability  density  parameter  [>  which  is  the 
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expected  number  of  target  encounters  per  unit  area.  The  Poisson  probability  parameter 
At,  is  then 

At  =  P A  ,  (6) 

and  the  probability  of  encountering  exactly  t  targets  in  A  becomes 

P(T  =  t)  =  e~^  j?-,  t  =0,1,2...  (7) 

Probability  of  Target  Report 

As  a  UCAV  searches  As  it  will  start  to  encounter  targets  and  false  targets  and 
execute  an  ATR  process  to  classify  them.  The  probability  of  the  ATR  process  correctly 
reporting  an  encountered  target  as  a  target  can  be  denoted  as  Pm  and  the  probability  of 
incorrectly  reporting  a  target  is  (1-Ptr)-  The  comparable  notation  for  the  ATR 
declaration  of  an  encountered  false  target  is  Pftr  and  (1-Pftr)- 

Confusion  Matrix 

For  the  simple  case  of  a  single  target  type,  Jacques  and  Pachter  [7]  modeled  the 
ATR  process  in  terms  of  a  binary  confusion  matrix,  shown  in  Table  1. 


Object  Encountered 

Object 

T 

FT 

T 

Ptr 

1-P FTR 

Reported 

As: 

FT 

1-Ptr 

Pftr 

Table  1.  Basic  Confusion  Matrix 
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Ideally  the  confusion  matrix  will  take  the  form  of  the  identity  matrix  where 
PTR  =  PFTR  =  1 .  This  infers  no  incorrect  target  declarations  can  occur.  It  is  more  realistic 

to  define  the  values  for  PTr  as  0  <  Ptr  <  1  and  Pftr  as  0  <  Pftr  <1  to  account  for  the 
likelihood  of  target  declaration  errors.  Overall,  the  confusion  matrix  implies  there  are 
three  possible  wide  area  search  and  engagement  outcomes  for  any  given  ATR  report: 

•  A  target  is  encountered  and  declared  a  target  resulting  in  a  Target  Attack 

0 TA ). 

•  A  false  target  is  encountered  and  declared  a  target  resulting  in  a  False 
Target  Attack  (FTA). 

•  A  false  target  is  encountered  and  declared  a  false  target  or  a  target  is 
encountered  and  declared  a  false  target  resulting  in  no  attack. 

Probability  of  Target  Attack 

Scenario  1:  Single  Warhead  UCAV 

For  the  single  warhead  UCAV  case  of  Scenario  1,  the  probability  of  encountering 
a  target  in  the  sensor  footprint  AA  is  predicated  on  the  probability  that  the  target  is  located 
in  A  A  and  the  probability  that  no  false  target  attack  occurred  previously  in  area  A.  The 
probability  of  a  target  encounter  can  be  written  as 

A  A 

PAM)  =  —  PjrAA  (8) 

As 

where  P—  (A)  represents  the  probability  of  no  false  target  attacks  occurring  prior  to 
arriving  at  area  AA.  The  probability  of  a  target  attack  is  then  a  matter  of  an  encountered 
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target  being  correctly  reported  as  represented  by  Ptr ■  Ptr  is  either  experimentally 
determined  or  can  be  arbitrarily  defined.  Equation  8  becomes 


Pta(M)  =  —Pw-M)Pt,-  (») 

Jacques  and  Pachter  [7]  showed  that  for  a  Poisson  field  of  false  targets  P—  (A)  is 
the  simple  case  of  k=0  for  Equation  5  and  can  be  rewritten  as 

P—(A)  =  e^-PFTR)aA.  (10) 

Substituting  Equation  10  into  Equation  9  gives 

PTA(AA)  =  ^-Pme-(1-p^aA.  (11) 

As 

Let  / (?)  be  the  probability  density  function  ip.d.f.)  for  a  target  attack  event 

occurring  at  time  t,  0<t<Ts  and  f(t)dt  be  the  probability  of  an  attack  occurring  over 

the  time  interval  [?,  t+dt].  Also,  recall  from  Figure  2  that  AA=WVdt.  Equation  1 1  can 
then  be  rewritten  as 

f(t)dt=J^PTRe~('-PFT*)aA.  (12) 

As 

The  p.d.f  for  a  target  attack  event  can  now  be  written  as 

f(t)  =  ^PTRe^aA.  (13) 

As 

Since  As  =  WVTS  and  A  =  WVt  then 

A  =  (14) 

A  c 
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and 


WV  _  1 
Ts 


From  Equation  4  the  Poisson  parameter  for  area  As  is 


"j  cc  *  J  y  , 


—  • 


Substitute  Equations  15  and  17  into  Equation  13  and  the  target  attack p.d.f,  can  be 


rewritten  as 


[  -(l-P t?TR  )aft  — 

f(t)  =  —PTRe 


In  a  similar  process,  Jacques  and  Pachter  [7]  derived  the  p.d.f  for  a  false  target 
attack  event  at  time  r,  0  <  t  <  Ts  as 


g’(Z")  PfTr)  1  P/'R  ^ 


(1  PrTR  )/.r 


The  probability  of  a  target  or  false  target  attack  occurring  in  As  can  be  written  as 
the  integral  of  the  p.d.f.  through  the  whole  battle  space  or 

Pta  =  £'  / {t)dt  and  PFTA  =  fj  g(r)dT  . 

Integrating  Equations  18  and  19  gives  the  closed  form  analytic  expressions  for 


Pta  and  Pfta  as 
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(20) 


P  =P 

1  TA  1  TR 


l-e 


-(l-P 'FTR  )XF 


(l  Pftr)^-ft 


and 


>3 

_ 1 

[  ( 

1  ^FTRJ^FT  J 

- 

-I-  P  P 
-r  i  TRe 


-(l-P 'FTR  )Af 


(21) 


The  probability  of  no  attack  occurring  is  then 

1  ■ - (Pta  +  Pm )  =  0 1  ■ -  P„ ) .  (22) 


Scenario  2:  Single  Warhead  UCAV 

For  the  single  warhead  UCAV  case  of  Scenario  2,  Jacques  and  Pachter  [7]  derived 
the  p.d.f.  equations  for  target  attack  and  false  target  attack  as 

]  ~[0  PFm  )  -V/  +  Put  -V  J=- 

=  -PTRATe  5  (23) 

1 s 

and 


S’(z")  (l  Pftr)Af Te 


[  ( '  PFTR  )-W  +  P//F-J  ].. 


(24) 


Integrating  Equations  23  and  24  over  As  gives  Pta  and  Pfta  as 


P  = 

1  TA 


Ptr^t 


( 1  PpTR  )  Apr  +  PtrAj 


-[(l-P C-77J  )Aft  +P rRAf  ] 


(25) 


and 


P  = 

1  FTA 


(  PpTR  )  ApT 


(l  PpTR  )  Aft  +  PtrAt 


l-e 


-[(1_-*F7X  )^FT  +PTR^T  ] 


(26) 
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Scenario  2:  Multi  Warhead  UCAV 

Decker  [1]  expanded  on  Jacques’  and  Pachter’s  equations  to  consider  a  multiple 
warhead  UCAV.  He  developed  expressions  for  the  probabilities  of  a  specific  number  of 
target  and  false  target  attacks  given  a  single  UCAV  searching  As  armed  with  w  warheads. 
In  these  expressions  Decker  utilized  a  Poisson  parameters  for  the  density  of  target  and 


false  target  attacks  defined 


2  —  p  2 

Aq'A  J'TR'^T  ? 


ApTA  (l  PfTr)\ 


Decker’s  expressions  for  PTa  and  Pfta  are  written  as 


TAX )  {^FTAX)  -{XrA  +XFTA  ).< 


p  _  'ST'  V^FTA^J  - 

TA=t\W~  Zj  tl  p  e 


o  t  o  w—t 
ATAAFTA 


w  1  /  ( (  AjA  +  Afta  )  x ) 

tJ(ATA+AFTA )  r(w) 


1  FTA=f\w 


f'  1  (AtaX)  ( AftaX )  c~(^A+XFTA)x  + 

[  t\  /! 

A'tAj Afta  (w\y{w’{.Ata+Afta)x} 
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In  Equations  27  and  28  above  r(w)  is  the  Gamma  function  which  can  be  written  as 

r(w)  =  (w-i)!, 

and  y(w, ( Xta+Xfta)x)  is  the  incomplete  Gamma  function  which  can  be  written  as 

y{w,{ATA+  ^fta)x)  =  £  ”  m)  . 

Decker  also  derived  equations  for  the  expected  number  of  target  and  false  target 
attacks  as 


t- 


f= 0 

W-1  w-t-\ 

ZZ 

/= 0  t=0 


KAZ  <  ^ 


vO 


{A TA  A FT a) 


7  (w,  ( ArA  +  AFrA)x^ 


r(w) 


+ 


{A TAX )  FTAX )  -(XrA  +XFTA  )x 


t\ 


/! 


and 


£[/]=! 


/ 


/ 

'FTA 


f= 0 

w-1  w-/- 1 

zz 

f=0  f=0 


v/y 


y(w,[ATA  +  Afta}x} 


FW 


/ 


f! 


/! 


(29) 


(30) 


Control  Problem  Formulation 


Receiver  Operating  Characteristic 

Before  the  control  problem  formulation  can  be  presented,  the  concept  of  a 
Receiver  Operating  Characteristic  (ROC)  must  be  discussed.  The  control  inputs  used  in 
the  control  problem  formulation  are  adapted  from  a  ROC  curve.  It  is  widely 
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acknowledged  in  sensor  signal  processing  theory  that  the  probabilities  for  correct  target 


classifications  and  false  positive  errors  [i.e.  Ptr  and  (1-Pftr)]  are  not  independent  and  can 
be  modeled  in  terms  of  a  ROC  curve.  Although  the  ROC  curve  is  determined 
experimentally,  a  mathematical  model  of  the  curve  can  be  used  analytically  relate  Ptr  and 
{1-Pftr}-  Kish  [8]  adapted  the  following  ROC  curve  equation  as  a  model  for  a  generic 
wide  area  search  and  engagement  sensor/ ATR  system. 


(31) 


A  plot  of  both  Kish’s  and  Decker’s  ROC  curve  equations  with  Ptr  on  the  y-axis 
and  (7-  Pftr)  on  the  x-axis  will  starts  at  (0,  0)  and  monotonically  increase  to  (1,  1). 
Figure  3  shows  a  family  of  ROC  curves  as  defined  by  Equation  (3 1). 


c  =  100 
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Figure  3.  Family  of  ROC  Curves 
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Decker  [1]  used  an  alternate  ROC  curve  equation  with  a  sensor  performance 
parameter  q. 

PFTR  =  1 _  Pm  (32) 

In  Equation  31,  the  scalar  parameter  c,  1  <  c  <  oo,  (or  q  in  Equation  32)  is 
dependent  on  factors  such  as  UCAV  velocity  and  altitude.  It  may  also  be  limited  by  the 
quality  of  the  sensor,  performance  of  the  ATR  algorithms  or  even  aspect  angle  of  the 
sensor  to  the  target.  The  value  of  c  determines  which  curve  in  a  family  of  ROC  curves 
characterizes  a  sensor/ATR  system’s  performance.  As  can  be  seen  in  Figure  3,  larger 
values  for  c  (or  q)  allow  for  higher  values  of  Ptr  before  ( 1-Pftr )  undergoes  any 
significant  rate  of  increase.  Kish  proposed  that  by  varying  UCAV  velocity  and  altitude,  c 
can  be  utilized  as  a  time  varying  control  input  to  optimize  ATR  performance.  In  both 
Equations  3 1  and  32,  the  value  of  Ptr  detennines  the  specific  position  on  a  given  ROC 
curve  that  a  sensot/ATR  system  operates  from  relative  to  {1-Pftr)-  The  parameter  Ptr 
can  then  be  used  as  a  control  input  to  strike  a  balance  between  Pja  and  Pfta-  This  is  the 
basis  for  the  control  problem  formulation. 

Basic  Problem  Formulation 

Kish  [8]  developed  a  generalized  optimization  control  problem  framework  for 
each  of  Jacques’  and  Pachter’s  six  academic  scenarios  that  maximized  Pja  subject  to  a 
constrained  Pfta-  The  basic  optimization  problem  statement  is  as  follows: 

Maximize:  PTA 

c  and/or  PTR 

Subject  to:  PFTA  <  PFTA 
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The  constraint,  PFTA  is  set  by  system  engineers  or  the  warfighter.  The  ROC  curve 

theory  above  provides  the  control  inputs  c  which  represents  a  UCAV’s  area  coverage 
rate  and  Ptr  which  represents  the  system  sensor  threshold.  Kish’s  control  problem 
formulations  used  the  following  three  control  input  cases: 

•  Fixed  Area  Coverage  Rate,  Fixed  Sensor  Threshold 

•  Fixed  Area  Coverage  Rate,  Variable  Sensor  Threshold 

•  Variable  Area  Coverage  Rate,  Variable  Sensor  Threshold 

Rosario  [12]  adapted  Kish’s  work  into  the  following  formulation  for  a  discrete¬ 
time  dynamic  optimization  problem  for  the  single  warhead  UCAV  case  of  Scenario  1 . 

The  control  inputs  are  a  fixed  area  coverage  rate  c  and  variable  sensor  threshold  Ptr ■  The 
problem  statement  becomes 

Subject  to:  s(z  +  l)  =  /[s(z),w(z),z] 
i^[5(  A)]  =  PFTA  <  PFTAm 


where  s(i+l)  represents  a  state  vector  of  equations 


x 


(,+1) 

v(/  +  l) 


and  u(i)=P 


TR- 


From  Equation  20  for  Pja  the  objective  function  becomes 

N 

^[x(A)]  =  (33) 


i=i 


From  Equation  2 1  the  constraint  equation  is 

A FTu  ( /  —  1) 


(l-c)zz(z -l)  +  c 


[l-y(z-l)]e‘Vjc(MW  (34) 
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The  state  equations  x(i+l)  and y(i+l)  are  pulled  from  the p.d.f.  Equations  18  and 
19  for  target  and  false  target  attacks  through  a  series  of  mathematical  manipulations.  For 
greater  detail  on  this  derivation  refer  to  [12].  For  simplicity  we  can  say  the  state 
equations  are  an  integration  of  the  ROC  curve  based  values  for  PTr  and  (1-Pftr)  over  the 
region  As  written  as 

x(/  +  l)  =  x(/)  +  - - Y^pr - A  t,  (35) 

(l-c)w(z)  +  c 

and 

y(i+l)  =  y(i)  +  u(i)At.  (36) 

For  comparison,  Rosario  also  developed  the  control  input  case  of  a  fixed  area 
coverage  rate  and  fixed  sensor  threshold.  This  problem  formulation  can  be  solved  using 
a  numerical  software  package  such  as  Mathworks  Matlab®. 

Chapter  Summary 

This  chapter  developed  the  analytic  background  theory  and  expressions  required 
to  conduct  a  thorough  validation  of  a  wide  area  search  and  engagement  computer 
simulation.  The  theory  and  expressions  presented  for  Scenario  2,  particularly  Equations 
27,  28,  29,  and  30  will  be  used  to  validate  the  ability  of  MultiUAV  to  realistically 
simulate  the  perfonnance  of  an  ATR  process  with  a  multiple  warhead  capable  UCAV 
(Research  Objective  1).  The  control  problem  formulation  developed  by  Rosario  [12]  as 
presented  above  will  be  used  to  evaluate  the  ability  of  MultiUAV  to  incorporate  time 
varying  parameters  to  dynamically  optimize  ATR  performance  (Research  Objective  2). 
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Chapter  3  will  provide  a  brief  overview  of  the  MultiUAV  simulation 
environment,  the  key  simulation  functions  used  to  model  the  ATR  process,  and 
modifications  to  the  simulation  required  to  implement  the  above  academic  scenarios  to 
accomplish  the  research  objectives. 
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III.  MultiUAV  Simulation  Environment 


Chapter  Overview 

Computer  simulation  is  an  essential  tool  for  the  research  and  development  of 
cooperative  control  algorithms  to  optimally  employ  multiple  UCAVs  in  wide  area  search 
and  engagement.  To  be  representative  of  real  world  conditions,  a  simulation  must  be  able 
to  accurately  model  the  performance  of  the  ATR  algorithms  that  will  be  used  to 
discriminate  between  targets  and  false  targets.  To  validate  this  ability,  the  outcomes  of 
basic  wide  area  search  and  engagement  scenarios  implemented  in  a  simulation 
environment  like  MultiUAV  may  be  compared  to  the  solutions  from  analytic  expressions 
such  as  those  presented  in  Chapter  2. 

This  chapter  presents  a  basic  overview  of  the  core  MultiUAV  simulation 
environment  and  the  embedded  simulation  functions  most  relevant  to  modeling  a  wide 
area  search  and  engagement  ATR  process.  These  functions  include:  the  battle  space 
setup,  how  targets  and  false  targets  are  distributed  in  the  simulated  battle  space,  the 
sequence  of  events  from  target  discovery  to  kill,  and  most  importantly  the  methodology 
used  to  classify  encountered  targets  and  false  targets.  In  addition,  this  chapter  describes 
modifications  made  to  theses  functions  as  well  as  some  additions  to  MultiUAV  to  enable 
simulations  of  the  academic  Scenarios  1  and  2  as  described  in  Chapter  2.  These 
modifications  include:  the  ability  to  accurately  simulate  Poisson  fields  of  targets  and  false 
targets,  the  implementation  of  the  confusion  matrix  and  ROC  curve  equations  to  model 
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an  ATR  process  and  finally  the  use  of  time  varying  parameters  as  control  inputs  to 
optimize  ATR  performance. 

MultiUAV  Simulation  Description 

AFRL/VACA  developed  MultiUAV  specifically  to  analyze  cooperative  control 
algorithms  for  use  in  autonomous  wide  area  search  and  engagement  applications. 
MultiUAV  is  built  on  the  Mathworks  Matlab®  and  Simulink®  programming 
environments  [11]  as  well  as  C++  functions.  It  is  an  open  source  simulation  tool  that  has 
been  adapted  for  use  in  numerous  ongoing  DoD,  industry  and  academic  research  studies 
for  cooperative  behavior  of  autonomous  vehicles. 


Figure  4.  MultiUAV  Simulation  Output  Plot 

The  basic  simulation  run  consists  of  multiple  UCAVs  searching  a  defined  area  in 
a  prescribed  pattern  for  a  given  quantity/density  of  targets.  The  user  can  modify 
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simulation  parameters  such  as  quantity  of  UCAVs  and  warheads,  arrangement  and  type 
of  targets  and  false  targets,  and  level  of  cooperative  behavior.  The  typical  MultiUAV 
simulation  run  will  produce  an  output  plot  such  as  the  one  shown  in  Figure  4  [11],  that 
shows  the  UCAV  paths  and  target  locations. 

Simulation  studies  of  specific  scenario  designs  are  executed  using  a  Monte  Carlo 
method  to  produce  statistically  significant  results.  Each  Monte  Carlo  run  uses  an 
incrementing  schedule  of  pseudo-random  numbers  to  determine  target  locations  and 
performance  measures  that  can  be  compared  to  various  threshold  criteria. 

Simulation  Functions 

For  this  research  the  MultiUAV  functions  of  greatest  concern  involve  how  targets 
and  false  targets  are  distributed  in  the  simulated  battle  space,  the  sequence  of  events  from 
target  discovery  to  kill,  and  most  importantly  the  ATR  process  used  to  classify 
encountered  targets  and  false  targets  is  implemented. 

Battle  Space 

The  basic  MultiUAV  battle  space  is  rectangular  in  shape  with  dimensions 
specified  according  to  the  desired  scenario  parameters.  The  battle  space  is  searched  in  a 
back  and  forth  “mowing  the  lawn”  pattern.  For  this  research  the  battle  space  illustrated  in 
Figure  2  is  employed.  The  width  of  the  battle  space  is  specified  to  be  equal  to  the  sensor 
swath  width  of  the  UCAV.  The  speed  of  the  UCAV  is  held  constant  and  the  duration  of 
all  simulation  runs  are  identical.  The  length  of  the  battle  space  can  be  defined  as  the 
velocity  of  the  UCAV  multiplied  by  the  total  simulation  run  time.  No  UCAV 
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maneuvering  is  allowed.  While  this  represents  a  greatly  simplified  simulation  scenario,  it 
will  allow  for  direct  comparison  with  the  analytic  expressions  discussed  in  Chapter  2. 

Target  Distribution  and  Target  Discovery  Times 

The  MultiUAV  simulation  allows  for  targets  to  be  arranged  in  the  battle  space 
according  to  a  uniform,  normal,  or  pseudo-Poisson  distribution  algorithm  depending  on 
the  desired  scenario  parameters.  Targets  are  inserted  into  the  battle  space  by  means  of  a 
numeric  array  of  target  discovery  times  that  is  generated  using  one  of  the  distribution 
algorithms.  In  the  case  of  the  unifonn  distribution,  the  target  discovery  time  array  is 
generated  using  the  Matlab  random  number  generator  command  RAND(N)  where  N  is  the 
number  of  desired  targets.  The  RAND(A)  command  produces  an  array  of  A  random 
numbers  with  values  between  0  and  1 .  The  array  is  scaled  to  the  simulation  environment 
by  simply  multiplying  each  element  in  the  array  by  the  total  simulation  run  time.  As  the 
simulation  runs  individual  targets  are  placed  in  front  of  the  sensor  footprint  of  the  UCAV 
according  to  the  discovery  times  in  the  array.  Therefore,  the  spatial  coordinate  of  the 
targets  are  directly  linked  to  that  of  the  UCAV  at  the  target  discovery  times.  Because  the 
UCAV  moves  through  the  simulated  battle  space  in  a  straight  line  and  constant  speed  this 
method  accurately  creates  the  desired  target  spatial  distribution  in  the  battle  space. 

Target  State  Machine  and  UCAV  Task  Assignments 

As  a  UCAV  searches  the  virtual  battle  space  it  will  start  to  encounter  targets.  The 
target  state  machine  shown  in  Figure  5  [1 1]  illustrates  the  target  “kill  chain”  from  target 
detection  to  verified  kill. 
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Figure  5.  MultiUAV  Target  State  Machine 

The  possible  target  states  assigned  by  the  simulation  to  each  target  are: 

•  Target-Not-Detected 

•  Detected-Not-Classified 

•  Classified-Not- Attacked 

•  Attacked-Not-Killed 

•  Killed-Not-Verified 

•  Target-Verified-Killed 

Task  assignments  are  given  to  each  UCAV  in  the  battle  space  based  on  an 
expected  state  of  a  target  UCAV  is  engaging.  These  tasks  consist  of: 

•  Search 

•  Classify 

•  Attack 

•  Verify 
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The  Search  task  is  the  nominal  state  for  a  UCAV  that  has  not  encountered  any 
targets  or  been  assigned  to  perform  a  task  on  a  target  detected  by  some  other  UCAV.  In 
the  case  of  a  single  UCAV  simulation,  once  a  target  is  detected  the  task  assignment 
algorithm  assumes  the  kill  chain  will  be  completed.  All  tasks  required  to  complete  the 
kill  chain  are  immediately  assigned  to  the  UCAV  with  an  associated  completion  time. 

The  UCAV  completes  each  task  in  the  appropriate  sequence  shown  in  Figure  5  until  the 
target  has  been  verified  as  killed.  In  the  event  a  false  target  is  encountered  and  correctly 
classified,  the  Attack  and  Verify  task  assignments  are  effectively  ignored  and  not 
completed  as  assigned.  In  the  case  of  simulations  involving  multiple  UCAV’s,  the  task 
assignments  are  allocated  to  individual  UCAV’s  based  on  the  specified  cooperation 
algorithm. 

Automatic  Target  Recognition  (ATR) 

The  ATR  process  is  implemented  as  part  of  the  Classify  task  assignment.  When  a 
target  enters  the  sensor  footprint  of  a  UCAV  for  the  first  time  its  state  is  changed  from 
target-not-detected  to  detected-not-classified.  One  or  more  UCAVs  are  then  assigned  the 
task  to  classify  the  target.  As  shown  in  Figure  5,  the  outcome  of  a  classify  task  is 
dependent  on  the  parameter  Pc-  Pc  represents  the  probability  of  target  identification 
confidence.  A  value  for  Pc,  0  <  Pc  <1,  is  given  based  on  the  angle  of  encounter  between 
the  path  of  the  UCAV  and  the  orientation  of  the  target  as  shown  in  Figure  6  [11]. 
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Figure  6.  UCAV  Angle  of  Encounter  to  Target 

If  the  value  of  Pc  meets  or  exceeds  a  user-defined  ATR-sensor-threshold  (for 
example,  0.9  in  Figure  5)  the  target  state  is  changed  to  classified-not-attacked.  If  the 
value  of  Pc  is  less  than  the  threshold  the  target  state  remains  detected-not-classified  and 
additional  classify  tasks  are  executed.  The  individual  values  of  Pc  from  each  classify 
task  are  combined  into  one  value  until  the  threshold  criteria  is  met. 

Modifications  and  Additions  to  MultiUAV 

The  following  modifications  and  additions  were  made  to  MultiUAV  to  allow  for 
implementation  of  the  academic  scenarios  used  to  accomplish  the  research  objectives. 
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Target  Discovery  Times  Modification 

Although  MultiUAV  has  algorithms  to  generate  a  uniform  distribution  and 
approximate  a  Poisson  distribution  with  a  fixed  quantity  of  targets,  these  algorithms  were 
not  employed  for  this  research.  This  was  due  to  a  limitation  of  MultiUAV  that  only 
allowed  for  one  target  type  and  one  distribution  algorithm  to  be  used  in  generating  the 
target  discovery  time  array.  Recall  that  Scenario  1  required  a  single  target  with  a  uniform 
distribution  and  a  Poisson  field  of  false  targets.  To  overcome  this  obstacle  the  algorithm 
used  to  create  the  target  discovery  time  array  was  replaced  with  one  that  allowed  for 
multiple  function  calls  to  any  of  the  target  distribution  algorithms.  The  target  discovery 
time  array  is  built  by  merging  the  output  from  each  function  call  into  a  single  array  that  is 
then  sorted  and  scaled  to  the  total  simulation  run  time.  The  actual  Matlab  m-file  used  is 
available  in  Appendix  A. 

Poisson  Distribution  Modification 

In  the  current  version  of  MultiUAV,  the  approximation  for  a  Poisson  field  of 
targets  contains  an  exact  quantity  of  targets  or  false  targets  equal  to  the  Poisson 
parameters  described  by  Equations  4  and  6  in  Chapter  2.  This  is  inadequate  since  a 
Poisson  distribution  is  defined  by  an  expected  target  density  over  a  defined  area  or  time. 

It  should  be  expected  that  two  realizations  of  a  Poisson  field  of  targets  with  identical 
Poisson  parameters  will  contain  different  quantities  of  targets.  For  this  work  a  new 
algorithm  for  generating  a  Poisson  distribution  was  implemented  that  built  the  target 
discovery  time  array  using  a  simple  Poisson  process  algorithm.  In  a  Poisson  process  the 
intervals  between  arrival  times  are  exponentially  distributed  with  parameter  1  A,  where  X 
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is  the  Poisson  parameter.  The  new  algorithm  utilized  the  ability  of  Matlab  to  generate 
random  numbers  with  an  exponential  distribution  to  build  a  Poisson  distribution  of 
targets.  The  actual  simulation  subroutine  used  is  reproduced  below: 


ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo 

%  Subroutine  PoissonDiscoveryTimes 

ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo 

function  [PoissonDiscoveryTimes] =PoissonDistribution (ExpectedTargets ) 

%  This  function  is  called  by  TargetDiscoveryTimes . m 

-O 

%  Inputs: 

%  ExpectedTargets  -  Number  of  expected  target  encounters 
%  of  any  target  type 

-6 

%  Outputs : 

%  PoissonDiscoveryTimes  -  Target  discovery  times  as  determined 
%  through  a  modelled  Poisson  process 

%  with  characterized  by  ExpectedTargets. 

"6 

%  Note:  Requires  Matalb  RANDOM  toolbox.  If  RANDOM  toolbox 
%  not  available  use  the  following  line  of  code: 

%  TimeStep (i) =-log (rand) /Lambda; 

Lambda=ExpectedTargets ;  %  Poisson  parameter 

DiscoveryTimes (1 ) =random (’ Exponential 1 , 1/Lambda) ;  %  first  tgt  arrival  time 

i=l; 
stop=0 ; 

while  (DiscoveryTimes (i) <1)  &&  (stop==0), 

TimeStep=random (' Exponential ' , 1/Lambda) ;  %  next  arrival  time 
if  TimeStep  <  1-DiscoveryTimes  (i ) ,  %  check  next  arrival  time  <  1 

DiscoveryTimes (i+1 ) =DiscoveryTimes (i) +TimeStep;  %  build  arrival  array 
i=i+l; 

else  %  stop  if  next  arrival  would  exceed  1 
stop=l ; 

end 

end 

PoissonDiscoveryTimes=DiscoveryTimes ' ;  %  OUTPUT  to  TargetDiscoveryTimes 
Return 

ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo 
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Target  State  Machine  Modification 

The  focus  of  this  research  was  to  quantify  the  performance  of  the  simulated  ATR 
process  in  MultiUAV  for  comparison  to  analytic  models.  The  data  product  of  interest  is 
the  outcome  of  all  simulated  ATR  classification  events.  Tracking  the  state  of  each  target 
past  the  classified-not-attacked  state  as  described  by  Figure  5  was  not  necessary  and 
would  have  required  unnecessary  computer  processing  time.  In  fact,  the  previous  work 
of  Dunkel  and  Schulz  the  number  of  Monte  Carlo  simulation  runs  that  could  be 
performed  was  limited  by  available  computer  resources.  Schulz  was  only  able  to 
considered  100  Monte  Carlo  simulations  for  each  scenario  design.  Fewer  simulation  runs 
may  introduce  unacceptable  differences  between  the  mean  values  produced  from  the 
simulation  results  and  the  analytic  solutions.  This  is  apparent  in  the  results  presented  in 
Chapter  4.  To  allow  for  more  Monte  Carlo  simulation  runs  MultiUAV  was  modified  so 
that  once  a  target  was  classified  the  subsequent  tasks  were  never  performed.  This  is  the 
same  as  how  a  false  target  classification  is  handled.  With  this  modification,  significantly 
more  Monte  Carlo  runs  could  be  performed  while  requiring  significantly  less  computer 
processing  time. 

An  additional  modification  was  made  to  capture  the  required  simulation  metrics 
by  altering  the  target  state  classified-not-attack  to  included  the  sub-state  type(ntruth)- 
classified-type(n declared)  with  n  target/declaration  types.  As  previously  acknowledged, 
only  one  target  type  and  one  false  target  type  was  considered  resulting  in  the  following 
four  possible  target  states: 

•  Target-Classified-Target 
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•  Target-Classified-False-Target 

•  False-Target-Classified-False-Target 

•  False-Target-Classified-Target 

This  allowed  for  an  easy  way  to  track  the  simulated  ATR  classification  results. 

All  simulation  classification  events  were  recorded  in  a  data  file  for  later  comparison  to 
the  analytical  solutions  of  Chapter  2. 

ATR  Confusion  Matrix  Modification 

In  the  earlier  versions  of  MultiUAV,  the  probability  of  correctly  identifying  an 
encountered  target  type  was  dependent  only  on  the  number  of  Classify  tasks  performed, 
increasing  the  value  of  Pc  until  the  threshold  criteria  was  met.  A  correct  target 
classification  was  guaranteed  provided  the  simulation  time  did  not  run  out  before  the 
value  of  Pc  exceeded  the  threshold  value.  This  does  not  accurately  duplicate  the 
performance  of  a  realistic  ATR  system. 

MultiUAV  was  modified  by  Dunkel  [3]  to  incorporate  a  confusion  matrix  into  the 
ATR  decision  model  to  allow  incorrect  target  declarations,  as  described  in  Chapter  2.  His 
modification  allowed  the  user  to  implement  the  Classify  task  with  either  the  Pc  values  as 
described  above  or  his  confusion  matrix  model  for  ATR  performance.  The  confusion 
matrix  used  in  MultiUAV  is  similar  to  the  one  in  Table  1  only  expanded  to  allow  for 
some  number  of  n  target  types  as  shown  in  Table  2  (n  =  5).  Any  target  type  could  be 
implemented  as  a  false  target.  Note  that  the  column  for  each  target  type  must  sum  to  1 . 
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Target  Type  Encountered 


T3 
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in 

O 

c. 
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H 


Type  1 

Type  2 

Type  3 

Type  4 

Type  5 

Type  1 

P  TR  1\T1 

P  TR  1\T2 

Ptr  i\t3 

P  TR  1\T4 

P  TR  1\T5 

Type  2 

P TR  2\T1 

P  JH  2\T2 

P TR  2\T3 

P TR  2\T4 

P TR  2\T5 

Type  3 

P TR  3\T1 

P TR  3\T2 

P TR  3\T3 

P TR  3\T4 

Ptr3\ts 

Type  4 

P TR  4\T1 

P TR  4\T2 

Ptr  4\  r 3 

P TR  4\T4 

P TR  4\T5 

Type  5 

P TR  5\T1 

P TR  5\T2 

Ptrs\t3 

P TR  S\T4 

P  TR  j|rj 

Table  2.  MultiUAV  Confusion  Matrix 


For  this  research  only  two  target  types,  a  generic  target  and  false  target,  were 
considered.  The  MultiUAV  confusion  matrix  subroutine  is  shown  in  Appendix  B. 

ROC  Curve  Modification 

An  additional  modification  made  to  the  MultiUAV  was  to  mathematically  link  the 
values  of  Ptr  and  Pftr  in  terms  of  the  ROC  curve  equations  presented  in  Chapter  2.  The 
modification  allows  the  user  to  specify  a  value  for  Ptr  that  is  then  used  to  automatically 
populate  MultiUAV’s  confusion  matrix  used  to  simulate  the  ATR  process  for  each  target 
encounter.  The  input  value  of  Ptr  can  be  based  on  empirical  data  from  experimentation 
with  a  real  sensor/ATR  system  or  given  as  a  purely  academic  parameter.  For  this 
research  PTr  was  given  as  an  academic  parameter.  PFTr  is  calculated  using  one  of  two 
equations.  Equation  32  is  used  for  simulations  of  Scenario  2  that  were  compared  to 
Decker’s  analytic  expressions.  Equation  31  is  used  for  simulations  of  Scenario  1  that 
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allowed  time  varying  parameters.  Equation  3 1  can  be  algebraically  rearranged  to  make 


Pftr  an  explicit  function  of  Ptr. 


P  =1- 

1  FTR  1 


1  TR 


(1  C)^TR+C 


(36) 


Time  Varying  ATR  Parameters  Addition 

The  benefit  of  using  the  ROC  curve  equation,  Equation  36,  to  define  the 
confusion  matrix  parameters  in  MultiUAV  is  that  it  provides  the  vehicle  to  implement  the 
control  input  parameters  for  the  dynamic  optimization  problem  formulation  described  in 
Chapter  2.  These  parameters  are  the  area  coverage  rate,  which  detennines  c,  and  the 
sensor  threshold,  which  determines  Ptr-  Ptr  and  c  can  easily  be  redefined  as  the  time 
varying  inputs  Pm(t)  and  c(t),  where  t  is  some  target  discovery  time  0  <  t  <TS.  Pftr  can 
then  be  expressed  as  a  function  PFTR{PTR(t),c(t)}.  The  input  c  was  held  constant  for  this 
work. 


Pftr  {^tr  (0,c(0}  ^ 


Pjj) 

(i-c(t))pra(t)+c(t) 


(37) 


The  value  for  Ptr(0  at  a  target  discovery  time  t  can  be  input  into  MultiUAV  from 
a  lookup  table  constructed  from  Rosario’s  [12]  Matlab  numerical  dynamic  optimization 
algorithm.  For  each  target  discovery  time  the  value  for  PtrO)  is  pulled  from  the  table  and 
input  directly  into  the  confusion  matrix  before  a  target  classification  is  made.  An 
alternate  method  not  applied  in  this  research  is  to  represent  PtrO)  and/or  c(t)  as  closed 
form  equations  in  the  simulation.  The  simulation  subroutines  used  to  model  the  ROC 
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curve  equations  and  time  varying  control  inputs  as  well  as  build  the  confusion  matrix  is 


reproduced  below: 


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 
%  Implements  Static  Ptr  using  Decker's  ROC  curve  equation 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 

Ptr  =0.9;  %  INPUT  -  Static  Ptr 

q  =  18;  %  INPUT  -  ROC  curve  equation  parameter 

Pftr  =  1  -  PtrAq;  %  Decker's  ROC  curve  equation  for  Pftr 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 
%  Implements  dynamic  Ptr  Schedule  with  Kish' s  control  problem 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 

load (’ PtrSchedule . mat 1 ) ;  %  Load  data  file  with  Ptr  schedule 

c  =  100;  %  INPUT  -  ROC  curve  equation  parameter 

lambda  =  20;  %  INPUT  -  Poisson  parameter  for  false  target  density 

DiscoveryTime= (TargetDiscoveryTimes (Target . ID)  ; 

Ptr  =  PtrSchedule (DiscoveryTime) ;  %  INPUT  -  Dynamic  control  input  Ptr 

Pftr  =  1-Ptr/ ( (1-c) *Ptr+c) ;  %  Kish's  ROC  curve  equation  for  Pftr 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 
%  Build  Confusion  matrix  for  True/False  targets 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 


ProbabilitylD (1,1) =Ptr; 

% 

Prob . 

of 

target 

report 

given 

target 

ProbabilitylD (2,1) =1-Ptr ; 

% 

Prob . 

of 

false 

target 

report 

given 

target 

ProbabilitylD (l,2)=l-Pftr; 

% 

Prob . 

of 

false 

target 

report 

given 

false  target 

ProbabilitylD (2,2) =Pftr; 

% 

Prob . 

of 

target 

report 

given 

false 

target 

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 


Chapter  Summary 

The  purpose  of  this  chapter  is  to  provide  an  overview  of  the  MultiUAV 
simulation  environment  and  how  the  academic  scenarios  developed  in  Chapter  2  were 
incorporated  to  accomplish  the  research  objectives.  The  key  simulation  functions 
pertaining  to  modeling  the  ATR  process  and  modifications  made  to  MultiUAV  were 
discussed. 
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Chapter  4  will  present  the  reduced  results  from  various  MultiUAV  simulations  of 
the  academic  scenarios  and  how  they  compare  to  the  analytic  expressions  from  Chapter  2. 
In  addition,  Chapter  4  will  provide  a  limited  analysis  on  how  results  from  these  types  of 
simulations  can  be  used  to  understand  design  considerations  in  the  development  of  wide 
area  search  and  engagement  UCAV  systems. 
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IV.  Results  and  Analysis 


Chapter  Overview 

This  Chapter  presents  the  reduced  results  from  various  MultiUAV  simulations 
and  how  they  compare  to  values  from  the  analytic  expressions  of  Chapter  2.  The 
parameters  for  each  simulation  scenario  were  developed  with  the  analytic  background 
theory  explained  in  Chapter  2  and  employed  using  the  capabilities  and  modifications  of 
the  MultiUAV  simulation  as  described  in  Chapter  3.  Extensive  post  simulation  data 
processing  was  required  to  convert  simulation  results  into  concise  graphical  and  tabular 
forms. 


Scenarios  Considered: 

Scenario  2  was  used  to  validate  the  ability  of  MultiUAV  to  simulate  the 
performance  of  an  ATR  process  with  a  multiple  warhead  capable  UCAV  for  use  in  wide 
area  search  and  destroy  research  applications  (Research  Objective  1).  Scenario  1  was 
used  to  evaluate  the  ability  of  MultiUAV  to  incorporate  time  varying  parameters  to 
dynamically  optimize  the  simulated  ATR  performance  for  wide  area  search  and 
engagement  research  applications  (Research  Objective  2). 

Simulation  Validation  for  a  Multiple  Warhead  Capable  UCAV 

The  reduced  data  presented  in  this  section  from  the  simulations  of  Scenario  2 
have  been  formatted  to  mimic  potential  trade  studies  that  may  be  performed  in  the  design 
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and  operational  application  of  an  autonomous  wide  area  search  and  engagement  UCAV 
system.  These  trade  studies  consist  of  the  following  cases: 

•  Case  1 :  Expected  number  of  attacks  vs.  probability  of  target  report. 

•  Case  2:  Expected  number  of  attacks  vs.  warhead  quantity. 

•  Case  3:  Probability  of  specified  number  of  attacks. 

Decker  [1]  suggested  that  these  trade  studies  can  be  used  to  make  design 
decisions  in  the  development  of  wide  area  search  and  engagement  UCAV  systems  as  well 
as  determine  the  resources  required  for  specific  operational  missions.  These  results  are 
used  to  validate  the  ATR  model  performance  of  MultiUAV. 

Simulation  Input  Parameters 

The  following  input  parameters  were  incorporated  into  MultiUAV  to  simulate  the 
various  studied  conditions  of  Scenarios  2: 

•  Probability  of  target  report  (constant)  Ptr. 

•  Sensor  threshold  parameter  q  from  Equation  32  to  calculate  Pftr 

•  Poisson  parameter  Xj  for  target  density  in  As. 

•  Poisson  parameter  Xfta  for  false  target  density  in  As. 

•  UCAV  warhead  quantity,  w. 

•  Number  of  Monte  Carlo  simulation  runs. 

Performance  Metrics 

For  Scenario  2  the  reduced  simulation  data  were  compared  to  the  following  four 
analytically  calculated  metrics  given  by  Equations  27,  28,  29,  and  30: 
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•  Probability  of  a  specified  number  of  target  attacks,  PtaO=N). 

•  Expected  number  of  target  attacks,  Eta- 

•  Probability  of  a  specified  number  of  false  target  attacks,  PFTA(f=M). 

•  Expected  number  of  false  target  attacks,  EFta- 

Case  1:  Expected  Number  of  Attacks  vs.  Probability  of  Target  Report 

Case  1  examines  the  ability  of  MultiUAV  to  accurately  simulate  the  ATR  process 
performance  of  a  wide  area  search  and  engagement  system  for  different  given  values  of 
PTr.  By  evaluating  ETA  and  EFTA  as  a  function  of  PTr  designers  may  detennine  the 
performance  level  required  to  accomplish  a  given  mission.  This  type  of  trade  study  may 
produce  potential  cost  savings  if  it  can  be  determined  a  less  sophisticated  and  assumedly 
cheaper  sensor/ ATR  system  can  satisfy  the  mission’s  performance  requirements. 

For  this  study,  two  sensor/ATR  system  types  were  considered  with  sensor 
threshold  values  of  q=l 8  and  q=10  from  Equation  32.  In  this  case  a  higher  value  of  q 
represents  a  better  sensor/ATR  system  while  the  lower  value  represents  a  less  capable  but 
cheaper  system.  The  distribution  of  targets  and  false  targets  in  the  battle  space  are 
defined  by  the  Poisson  parameters  XF=10  and  XFF=20.  The  simulated  UCAV  was  armed 
with  a  warhead  quantity  of  w=10.  The  values  used  for  PTr  varied  from  0. 1  to  1  in 
increments  of  0. 1 .  Because  of  computer  processing  time  limitations  the  results  presented 
here  were  reduced  from  100  Monte  Carlo  simulation  runs  for  each  value  of  PFr. 

The  anticipated  perfonnance  outcome,  as  shown  by  Decker  [1],  should  reflect  that 
for  low  values  of  PFr  the  value  of  ETA  for  both  sensor/ATR  systems  will  be  nearly 
identical.  This  is  because  for  low  values  of  Ptr  the  resulting  occurrence  of  false  positive 


42 


errors  is  negligible  as  determined  by  the  extremely  small  values  for  (1-Pftr)  calculated 
through  the  ROC  curve  equation,  Equation  32.  As  Pjr  is  increased,  the  value  of  ETA  for 
both  sensor/ ATR  systems  will  hit  a  peak  and  start  to  decline  as  false  positive  errors 
becomes  more  of  a  factor  with  increasing  values  of  (1-Pftr)  (he.  warheads  start  to  be 
used  on  false  targets).  It  is  expected  the  cheaper  sensor/ATR  system  will  hit  its  peak  Eta 
before  the  better  system.  As  the  value  of  Pm  goes  to  1  for  both  systems  false  positive 
errors  as  indicated  by  Efta  will  overwhelm  Eta  because  of  the  proportionally  higher 
density  of  false  targets  in  the  battle  space.  This  behavior  pattern  is  evident  in  Figures  7 
and  8  below  that  plot  the  simulation  results  and  analytic  solutions  of  Eta  and  Efta  as  a 
function  of  Pm  for  the  two  sensor/ATR  systems.  Tables  3  and  4  quantify  the  same 
results. 


Figure  7.  ETA  &  EFTa  vs.  Pjr  given  Xj =10,  XF7=20,  w=10,  q=18 
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Figure  8.  ETA  &  EFTa  vs.  PTr  given  /./=I0,  AF7=20,  w=10,  q=10 


Eta 

Efta 

Ptr 

PFTR 

Analytic 

Value 

Simulated 

Mean 

%  Diff. 

Analytic 

Value 

Simulated 

Mean 

%  Diff. 

0.10 

1.00 

1.000 

1.110 

0.110 

0.000 

0.000 

0.000 

0.20 

1.00 

2.000 

1.840 

0.080 

0.000 

0.000 

0.000 

0.30 

1.00 

3.000 

3.240 

0.080 

0.000 

0.000 

0.000 

0.40 

1.00 

3.996 

3.980 

0.004 

0.000 

0.000 

0.000 

0.50 

1.00 

4.978 

4.930 

0.010 

0.000 

0.000 

0.000 

0.60 

0.9999 

5.923 

5.540 

0.065 

0.002 

0.000 

0.000* 

0.70 

0.998 

6.794 

6.510 

0.042 

0.032 

0.040 

0.250 

0.80 

0.98 

7.487 

6.870 

0.082 

0.337 

0.340 

0.009 

0.90 

0.85 

7.077 

7.000 

0.011 

2.360 

2.540 

0.076 

1.00 

0.00 

3.333 

3.220 

0.034 

6.667 

6.780 

0.017 

Average  %  Diff. 

0.052 

Average  %  Diff. 

0.035 

*  Because  EFTa  is  extremely  small  the  %  dill',  is  considered  equal  to  0.000 


Table  3.  ETA  &  EFTA  vs.  PTR  given  Aj=10,  AFI=20,  w=10 ,  q=18 
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Eta 

E  ETA 

Analytic 

Value 

Simulated 

Mean 

%  Diff. 

Analytic 

Value 

Simulated 

Mean 

%  Diff. 

0.10 

1.00 

1.000 

1.290 

0.290 

0.000 

0.000 

0.000 

0.20 

1.00 

2.000 

1.990 

0.005 

0.000 

0.000 

0.000 

0.30 

1.00 

3.000 

2.780 

0.073 

0.000 

0.000 

0.000 

0.40 

0.9999 

3.996 

4.010 

0.004 

0.002 

0.000 

0.000* 

0.50 

0.999 

4.977 

5.060 

0.017 

0.019 

0.020 

0.053 

0.60 

0.99 

5.914 

5.990 

0.013 

0.119 

0.090 

0.244 

0.70 

0.97 

6.709 

6.520 

0.028 

0.541 

0.600 

0.109 

0.80 

0.89 

6.950 

6.870 

0.012 

1.866 

2.000 

0.072 

0.90 

0.65 

5.588 

5.650 

0.011 

4.330 

4.200 

0.030 

1.00 

0.00 

3.333 

3.420 

0.026 

6.667 

6.580 

0.013 

Average  %  Diff. 

0.048 

Average  %  Diff. 

0.052 

*  Because  E,  ta  is  extremely  small  the  %  dill',  is  considered  equal  to  0.000 


Table  4.  Eta  &  Epta  vs.  P tr  given  /.  / — 10,  //y — 20,  w—10,  q—10 


In  both  Figure  7  and  8  the  MultiUAV  simulation  results,  represented  by  the 
dashed  curves,  closely  follow  the  analytic  solutions  represented  by  the  solid  curves.  The 
analytic  solution  falls  within  a  99%  confidence  interval  of  the  simulation  results  for  all 
simulation  runs.  In  Tables  3  and  4  the  largest  percent  difference  between  the  simulated 
and  analytic  solutions  is  29%  for  Eta  with  Ptr=0.1  and  q=10.  It  is  important  to  comment 
here  on  the  effect  of  a  low  number  of  Monte  Carlo  simulation  runs  has  on  the  consistency 
of  the  simulated  data.  In  light  of  the  data  presented  in  the  following  sections  for  Cases  2 
and  3  the  larger  percent  difference  values  shown  here  can  be  said  to  be  entirely  due  to  the 
relatively  low  number  of  Monte  Carlo  runs. 

A  potential  trade  study  of  the  results  above  may  recommend  the  cheaper 
sensor/ATR  system  if  low  values  of  Ptr  are  acceptable.  For  example,  with  a  Ptr  value  of 
0.5  (i.e.  only  half  of  encountered  targets  are  correctly  declared)  the  simulation  generated 
Eta  values  of  4.930  for  the  supposed  better  sensor/ATR  system  and  5.060  for  the  cheaper 
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system.  Note  the  analytic  values  of  ETA  for  the  two  systems  are  nearly  identical  at  4.978 
and  4.977.  In  this  case  Efta  is  negligible  for  both  systems.  For  all  practical  purposes  the 
two  systems  perform  at  the  same  level,  therefore,  the  cheaper  system  should  be  chosen  if 
cost  is  a  concern.  In  contrast,  for  a  Ptr  of  0.80  or  greater  the  values  of  Eta  are  marginally 
different  while  the  values  for  Efta  are  at  least  0.340  for  the  better  sensor/ATR  system  and 
2.000  for  the  cheaper  system.  For  higher  Ptr,  the  better  sensor  system  should  be  chosen. 

Case  2:  Expected  Number  of  Attacks  vs.  Warhead  Quantity 

Case  2  examines  the  ability  of  MultiUAV  to  simulate  the  expected  combat 
performance  of  a  wide  area  search  and  engagement  system  for  different  warhead 
quantities.  By  evaluating  Eta  and  Efta  as  a  function  of  warhead  quantity  designers  can 
better  understand  vehicle  size  requirements  to  accomplish  a  given  mission.  Obviously, 
more  warheads  equate  to  a  larger  and  more  expensive  UCAV.  In  addition,  this  type  of 
study  can  be  used  to  determine  the  number  of  warheads  and/or  UCAV’s  required  to 
accomplish  a  specific  operational  mission. 

Two  sensor  types  are  examined  here  with  the  same  sensor  threshold  values  q=18 
and  q=10  as  in  Case  1.  The  distribution  of  targets  and  false  targets  in  the  battle  space  is 
also  the  same  with  the  Poisson  parameters  Xt=10  and  Xft=20.  In  addition,  a  Ptr  value  of 
0.90  was  used.  Results  are  reduced  from  1000  Monte  Carlo  simulation  runs  for  each 
sensor  type. 

The  anticipated  perfonnance  outcome  is  that  the  better  sensor/ATR  system  should 
be  able  to  achieve  higher  Eta  values  and  lower  Efta  with  fewer  warheads  than  the 
cheaper  system.  Figures  9  and  10  below  plot  the  simulation  and  analytic  results  for  Eta 
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Expected  Attacks 


and  Efta  as  a  function  of  w  for  the  two  sensor/ATR  systems  modeled.  Tables  5  and  6 
quantify  the  same  results. 


Figure  9.  Eta  &  Efta  vs.  w  given  /./— /  0,  Apj—20,  P tr—  0.90,  q=18 
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Figure  10.  ETA  &  EFta  vs.  w  given  2,= 10,  /.,  i=20,  PTr=0.90,  q=10 


Eta 

Efta 

Warhead 

Qtv 

Analytic 

Value 

Simulated 

Mean 

%  Diff. 

Analytic 

Value 

Simulated 

Mean 

%  Diff. 

2 

1.500 

1.511 

0.007 

0.500 

0.489 

0.022 

4 

2.997 

3.010 

0.004 

1.000 

0.988 

0.012 

6 

4.476 

4.503 

0.006 

1.493 

1.466 

0.018 

8 

5.875 

5.906 

0.005 

1.959 

1.917 

0.021 

10 

7.076 

7.068 

0.001 

2.360 

2.315 

0.019 

12 

7.970 

7.947 

0.003 

2.658 

2.593 

0.024 

14 

8.527 

8.474 

0.006 

2.844 

2.791 

0.019 

16 

8.815 

8.769 

0.005 

2.940 

2.883 

0.019 

18 

8.938 

8.891 

0.005 

2.981 

2.927 

0.018 

Average  %  Diff. 

0.005 

Average  %  Diff. 

0.019 

Table  5.  Eta  &  Epta  vs.  w  given  /.  /—1 0,  XFI —20,  P tr—  0.90,  q=18 
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Eta 

Efta 

Warhead 

Qty 

Analytic 

Value 

Simulated 

Mean 

%  Diff. 

Analytic 

Value 

Simulated 

Mean 

%  Diff. 

2 

1.127 

1.150 

0.020 

0.873 

0.850 

0.026 

4 

2.254 

2.283 

0.013 

1.746 

1.717 

0.017 

6 

3.379 

3.377 

0.001 

2.619 

2.622 

0.001 

8 

4.498 

4.549 

0.011 

3.486 

3.439 

0.013 

10 

5.588 

5.650 

0.011 

4.330 

4.271 

0.014 

12 

6.598 

6.594 

0.001 

5.113 

5.095 

0.004 

14 

7.459 

7.415 

0.006 

5.780 

5.780 

0.000 

16 

8.113 

8.073 

0.005 

6.287 

6.279 

0.001 

18 

8.547 

8.532 

0.002 

6.623 

6.615 

0.001 

Average  %  Diff. 

0.008 

Average  %  Diff. 

0.009 

Table  6.  Eja  &  Epta  vs.  w  given  /./—1 0, 1F 3 — 20,  P/r—  0.90,  q=10 


The  MultiUAV  simulation  results  for  Eta  and  Efta  of  both  sensor/ATR  systems 
are  nearly  indistinguishable  from  the  analytic  solutions  as  shown  in  Figures  9  and  10. 

The  analytic  solutions  fall  within  a  99%  confidence  interval  of  the  simulation  results. 

This  strong  correlation  can  be  attributed  to  the  relatively  high  number  of  Monte  Carlo 
runs  as  compared  to  the  results  from  Case  1 .  Flere,  the  largest  percent  difference  between 
the  simulated  and  analytic  solutions  is  2.6%  for  Efta  with  q=10  and  a  warhead  quantity 
of  w=2.  This  is  negligible  when  considering  it  represents  a  2.6%  difference  in  a  measure 
of  an  expected  number  of  attacks. 

In  a  trade  study  the  better  sensor/ATR  system  appears  to  significantly  outperform 
the  cheaper  system.  The  difference  in  values  for  Efta  between  the  two  systems  is 
particularly  dramatic.  As  an  example,  a  weapon  load  of  eight  warheads  produces  an  Efta 
of  1.917  for  the  better  sensor/ATR  system  and  3.439  for  the  cheaper  system.  For  the 
same  load  the  values  of  Eta  is  5.906  for  the  better  system  and  4.549  attacks  for  the 
cheaper  one.  Overall,  it  is  apparent  that  the  better  sensor/ATR  system  should  be  chosen 
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if  there  is  a  specific  need  to  constrain  EFTA  or  limit  UCAV  size  as  a  function  of  warhead 
quantity. 

Case  3:  Probability  of  a  Specified  Number  of  Attacks 

Case  3  examines  the  ability  of  MultiUAV  to  provide  data  to  estimate  the 
probability  of  at  lease  some  specified  number  of  attacks  occurring  for  a  given  mission 
scenario.  As  in  Case  1,  this  evaluation  may  give  insight  into  the  level  of  ATR 
performance  required  to  accomplish  a  given  mission  and  produce  potential  cost  savings  if 
a  cheaper  sensor/ATR  system  can  meet  the  perfonnance  requirements.  In  addition,  this 
type  of  study  may  be  used  to  determine  the  number  of  warheads  or  UCAV’s  required  to 
accomplish  a  specific  operational  mission. 

The  same  two  sensor  types  are  examined  here  with  the  sensor  threshold  values  of 
q=18  and  q=10.  The  distribution  of  targets  and  false  targets  in  the  battle  space  is  again 
defined  by  the  Poisson  parameters  Xj=10  and  Xft=20.  The  warhead  quantity  specified  is 
w=10.  The  data  set  is  generated  from  1000  Monte  Carlo  simulation  runs  for  each  sensor 
type. 

The  anticipated  performance  is  that  the  better  sensor/ATR  system  will  maintain 
higher  values  of  Pta  as  the  specified  number  of  target  attacks  increases  and  lower  values 
of  Pfta  as  the  number  of  false  target  attacks  increases.  Figures  1 1  and  12  below  plot  the 
simulated  and  analytic  results  for  Pta  and  Pfta  as  a  function  of  at  least  N  target  or  M  false 
target  attacks  occurring  for  the  two  sensor/ATR  systems  modeled.  Tables  7,  8,  9  and  10 
quantify  the  same  results. 
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1 


Figure  11.  PAttack{TA  or  FTA>X}  given  a,=IO,  a,., =20,  Pm=0.90,  w=10,  q=18 


Figure  12.  PAttack{TA  or  FTA>X}  given  A/=IO,  a,., =20,  Pm=0.90,  w=10,  q=10 
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Possible 

N=TA 

Values 

Frequency 

of 

N=TA 

Pta{TA=N} 

Pta/TA  >N} 

Analytic 

Simulated 

%  Diff. 

Analytic 

Simulated 

%  Diff. 

0 

0 

0.000 

0.000 

0.000 

1.000 

1.000 

0.000 

1 

0 

0.001 

0.000 

0.001 

1.000 

1.000 

0.000 

2 

4 

0.005 

0.004 

0.001 

0.999 

1.000 

0.001 

3 

17 

0.017 

0.017 

0.000 

0.994 

0.996 

0.002 

4 

41 

0.043 

0.041 

0.002 

0.977 

0.979 

0.002 

5 

102 

0.094 

0.102 

0.008 

0.934 

0.938 

0.004 

6 

162 

0.170 

0.162 

0.008 

0.840 

0.836 

0.004 

7 

248 

0.239 

0.248 

0.009 

0.670 

0.674 

0.004 

8 

253 

0.239 

0.253 

0.014 

0.431 

0.426 

0.005 

9 

127 

0.149 

0.127 

0.022 

0.192 

0.173 

0.019 

10 

46 

0.043 

0.046 

0.003 

0.043 

0.046 

0.003 

Average  %  Diff. 

0.006 

Average  %  Diff. 

0.004 

Table  7.  Pta{TA>N}  given  a,=10,  a ,  ,=20,  Pm=0.90,  w=10,  q=18 


Possible 

M=FTA 

Values 

Frequency 

of 

M=FTA 

Pta{FTA=M} 

Pta{FTA  >M} 

Analytic 

Simulated 

%  Diff. 

Analytic 

Simulated 

%  Diff. 

0 

77 

0.072 

0.077 

0.005 

1.000 

1.000 

0.000 

1 

219 

0.210 

0.219 

0.009 

0.928 

0.923 

0.005 

2 

283 

0.286 

0.283 

0.003 

0.718 

0.704 

0.014 

3 

237 

0.236 

0.237 

0.001 

0.432 

0.421 

0.011 

4 

117 

0.130 

0.117 

0.013 

0.196 

0.184 

0.012 

5 

54 

0.050 

0.054 

0.004 

0.066 

0.067 

0.001 

6 

11 

0.013 

0.011 

0.002 

0.016 

0.013 

0.003 

7 

1 

0.002 

0.001 

0.001 

0.003 

0.002 

0.001 

8 

1 

0.000 

0.001 

0.001 

0.000 

0.001 

0.001 

9 

0 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

10 

0 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

Average  %  Diff. 

0.004 

Average  %  Diff. 

0.004 

Table  8.  Pfta{FTA>M}  given  a,=10,  a,  , =20,  Pm=0.90,  w=10,  q=18 
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Possible 

N=TA 

Values 

Frequency 

of 

N=TA 

Pta{TA=N} 

Pta/TA  >N} 

Analytic 

Simulated 

%  Diff. 

Analytic 

Simulated 

%  Diff. 

0 

0 

0.000 

0.000 

0.000 

1.000 

1.000 

0.000 

1 

0 

0.004 

0.000 

0.004 

1.000 

1.000 

0.000 

2 

19 

0.021 

0.019 

0.002 

0.996 

1.000 

0.004 

3 

54 

0.069 

0.054 

0.015 

0.975 

0.981 

0.006 

4 

148 

0.150 

0.148 

0.002 

0.906 

0.927 

0.021 

5 

240 

0.228 

0.240 

0.012 

0.756 

0.779 

0.023 

6 

249 

0.241 

0.249 

0.008 

0.528 

0.539 

0.011 

7 

186 

0.176 

0.186 

0.010 

0.287 

0.290 

0.003 

8 

78 

0.084 

0.078 

0.006 

0.111 

0.104 

0.007 

9 

22 

0.024 

0.022 

0.002 

0.027 

0.026 

0.001 

10 

4 

0.003 

0.004 

0.001 

0.003 

0.004 

0.001 

Average  %  Diff. 

0.006 

Average  %  Diff. 

0.007 

Table  9.  Pta{TA>N}  given  Xi=10,  Xf1=20,  Ptr=0.90,  w=10,  q=10 


Possible 

M=FTA 

Values 

Frequency 

of 

M=FTA 

PTA{FTA=M] 

Pta{FTA  >M} 

Analytic 

Simulated 

%  Diff. 

Analytic 

Simulated 

%  Diff. 

0 

4 

0.004 

0.004 

0.000 

1.000 

1.000 

0.000 

1 

25 

0.027 

0.025 

0.002 

0.996 

0.996 

0.000 

2 

84 

0.091 

0.084 

0.007 

0.969 

0.971 

0.002 

3 

193 

0.183 

0.193 

0.010 

0.878 

0.887 

0.009 

4 

254 

0.244 

0.254 

0.010 

0.695 

0.694 

0.001 

5 

236 

0.224 

0.236 

0.012 

0.451 

0.440 

0.011 

6 

142 

0.143 

0.142 

0.001 

0.227 

0.204 

0.023 

7 

45 

0.063 

0.045 

0.018 

0.084 

0.062 

0.022 

8 

17 

0.018 

0.017 

0.001 

0.021 

0.017 

0.004 

9 

0 

0.003 

0.000 

0.003 

0.003 

0.000 

0.003 

10 

0 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

Average  %  Diff. 

0.006 

Average  %  Diff. 

0.007 

Table  10.  Pfta{FTA>M}  given  Xf=10,  XFJ=20,  PTR=0.90,  w=10,  q=10 


Again,  the  MultiUAV  simulation  results  for  both  sensor/ ATR  systems  are  nearly 
indistinguishable  from  the  analytic  solutions  as  shown  in  Figures  1 1  and  12.  The  analytic 
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solutions  fall  within  a  99%  confidence  interval  of  the  simulation  results.  In  this  case  the 
largest  percent  difference  between  the  simulated  and  analytic  solutions  is  2.3%.  This  is 
negligible  when  considering  it  represents  a  2.3%  difference  in  the  probability  of  at  least 
some  number  of  attacks  occuring. 

The  better  sensor/ATR  system  appears  to  outperform  the  cheaper  system  as 
expected.  For  example,  the  probability  that  the  better  system  will  yield  at  least  six  target 
attacks  is  0.840  compared  to  0.528  for  the  cheaper  one.  The  probability  that  the  better 
sensor  system  will  results  in  at  least  six  false  target  attacks  is  0.016  for  the  better  system 
and  0.227  for  the  cheaper  one.  This  type  of  trade  study  should  conclude  the  better  ATR 
system  is  preferred  if  there  is  a  specific  need  for  higher  values  of  Eta  for  a  mission  or 
there  is  a  requirement  to  constrain  Efta- 

Simulation  Evaluation  for  Time  Varying  Control  Parameters 

The  reduced  data  presented  in  this  section  are  from  MultiUAV  simulations  that 
incorporated  time  varying  control  input  schedules  for  Ptr.  The  time  scheduled  values  for 
Pm  were  obtained  from  Rosario’s  [12]  numeric  Matlab  dynamic  optimization  as 
presented  in  Chapter  2.  The  following  Scenario  1  variations  were  considered: 

•  Case  la:  Maximize:  Pta,  Subject  to:  Pfta  <  1-0,  c=100,  XFT=25 

•  Case  lb:  Maximize:  Pta,  Subject  to:  Pfta  <  0.2,  c=100,  Xft=25 

•  Case  2a:  Maximize:  Pta,  Subject  to:  Pfta  <  1-0,  c=100,  Xft=5 

•  Case  2b:  Maximize:  Pta,  Subject  to:  Pfta  <  0.2,  c=100,  aft=5 

•  Case  3a:  Maximize:  Pta,  Subject  to:  Pfta  <  1-0,  c=50,  lFT=25 
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•  Case  3b:  Maximize:  PTa,  Subject  to:  PFTa  <  0.2,  c=50,  XFt=25 

•  Case  4a:  Maximize:  Pta,  Subject  to:  Pfta  <  10,  c=50,  Xft=5 

•  Case  4b:  Maximize:  Pta,  Subject  to:  Pfta  <  0.2,  c=50,  XFT=5 

For  each  of  the  above  cases  studied  a  parallel  static  optimization  simulation  is 
presented  as  a  basis  for  comparison.  The  static  optimization  formulation  provides  a 
constant  value  of  Ptr  that  is  intended  to  maximize  Pta  subject  to  the  same  constrained 
values  of  Pfta ■  The  static  values  for  PTr  were  also  obtained  from  Rosario’s  [12]  Matlab 
optimization. 

Simulation  Input  Parameters 

The  following  input  parameters  were  required  by  MultiUAV  to  allow  dynamic 
and  static  optimization  simulations  of  Scenario  1 : 

•  Probability  of  target  report  (static  and  time  varying)  Pm(t). 

•  Sensor  threshold  parameter  c  from  Equation  3 1  to  calculate  Pftr 

•  Poisson  parameter  Xfta  for  false  target  density  in  As. 

•  Number  of  Monte  Carlo  simulation  runs. 

The  maximum  allowed  constraint  Pfta  values  were  chosen  to  model  an 
unconstrained  optimization  case  with  Pfta=1-0  and  a  highly  constrained  case  with 
Pfta=0.20.  The  values  for  the  ROC  curve  equation  parameter  c  in  the  case  studies 
defined  above  were  chosen  to  model  a  high  quality  sensor/ATR  system  with  c=100  and  a 
low  quality  system  with  c=50.  The  values  of  Xft  were  chosen  to  model  a  high  false  target 
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density  in  the  battle  space  with  1FT  =25  and  a  low  target  density  with  1ft  =5  for  each 
sensor/ATR  system. 

Performance  Metrics 

For  Scenario  1,  the  reduced  simulation  data  were  compared  to  the  numeric 
dynamic  and  static  optimization  output  data  from  Rosario’s  [12]  Matlab  algorithm 
consisting  of: 

•  Probability  of  target  attack,  Pta- 

•  Probability  of  false  target  attack,  Pfta- 

It  is  expected  that  the  simulations  implementing  time  varying  PTr  schedules  will 
produce  higher  final  values  of  Pta  than  simulations  using  static  Pjr  values.  It  is  also 
expected  that  the  unconstrained  simulations  (i.e.  PFTa<1-0)  will  produce  higher  final 
values  of  both  PTA  and  PFta  than  the  constrained  simulations  ( PFFa<0.2 ). 

The  results  from  each  MultiUAV  simulation  are  presented  in  Figures  13-28  and 
quantified  in  Tables  1 1  and  12  in  the  following  pages. 
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Case  la:  Maximize:  PTa>  Subject  to:  Pfta  <  1-0,  c=100,  /.Ff=25 
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Figure  13.  Dynamic  Optimization:  PFTA  <  1.0,  c=100,  =25 
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Figure  14.  Static  Optimization:  PFTa  <  1-0,  c=100,  AF1=25 
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Case  lb:  Maximize:  PTa,  Subject  to:  Pfta  <  0.2,  c=100,  AF7=25 
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Figure  15.  Dynamic  Optimization:  Pfta  <  0.2 ,  c=100 ,  kF1 =25 
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Figure  16.  Static  Optimization:  PFTa  <  0.2 ,  c=100,  kF1=25 
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Case  2a:  Maximize:  PTa,  Subject  to:  Pfta  <  1.0,  c=100,  AF7=5 
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Figure  17.  Dynamic  Optimization:  Pfta  <  1»0,  c=100,  XF1 =5 
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Figure  18.  Static  Optimization:  PFTA  <  1.0,  c=100,  kF1 =5 
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Case  2b:  Maximize:  PTa,  Subject  to:  Pfta  <  0.2,  c=100,  AF7=5 
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Figure  19.  Dynamic  Optimization:  PFT4  <  0.2,  c=100,  XFJ =5 
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Figure  20.  Static  Optimization:  PFTa  <  0.2,  c=100,  kF1 =5 
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Case  3a:  Maximize:  PTa,  Subject  to:  Pfta  <  1-0,  c=50,  aft=25 


o 

CL 

0 

DC 

0 

cn 


0.9  - 

0.8  - 

0.7 

0.6 

0.5 

0.4 

0.3 

0.2 


0.1 


- PTR  Schedule 

- Simulation  PJA 

Optimized  P 
—  Simulation  PFT/ 
□  Optimized  P 


0.2  0.4  0.6 

Normalized  Time  (x) 


0.8 


1 

0.9 

0.8 


0.7 -g 
0 

0.6  5 
'<5 

0.5  >, 

0.4  Hj 
J2 
P 

0.3  £ 


0.2 

0.1 

0 


Figure  21.  Dynamic  Optimization:  PFTa  <  1-0,  c=50,  XF1 =25 
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Figure  22.  Static  Optimization:  PFTa  <  1-0,  c=50,  /.Fi=25 
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Case  3b:  Maximize:  PTa,  Subject  to:  Pfta  <  0.2,  c=50,  aft=25 
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Figure  23.  Dynamic  Optimization:  PFT4  <  0.2,  c=50,  / Ft=25 
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Figure  24.  Static  Optimization:  PFTa  <  0.2,  c=50,  2FJ=25 
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Case  4a:  Maximize:  PTa>  Subject  to:  Pfta  <  1-0,  c=50,  aF7=5 
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Figure  25.  Dynamic  Optimization:  PFTa  <  /. 0,  c=50,  XF7 =5 
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Figure  26.  Static  Optimization:  PFTa  <  1-0,  c=50,  XF1 =5 
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Case  4b:  Maximize:  PTa>  Subject  to:  Pfta  <  0.2,  c=50,  aF7=5 


Normalized  Time  (x) 


Figure  27.  Dynamic  Optimization:  PFTa  <  0.2,  c=50,  XF7 =5 
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Figure  28.  Static  Optimization:  PFTa  <  0.2,  c=50,  XF1 =5 
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Dynamic  Max  PTA 

Dynamic  Final  PTFA 

Numeric 

Solution 

Simulated 

Results 

%  Diff. 

Numeric 

Solution 

Simulated 

Results 

%  Diff. 

Case  la:  c=100,  2F1=25,  PFT4  <1.0 

0.551 

0.552 

0.001 

0.325 

0.018 

Case  lb:  c=100,  XFT  ~25,  PFT4  <  0.2 

0.487 

0.496 

0.009 

0.172 

0.028 

Case  2a:  c=100,  2FT=5,  PFta  <  1.0 

0.759 

0.768 

0.159 

0.011 

0.759 

0.768 

0.159 

0.011 

0.459 

0.200 

0.685 

0.697 

0.012 

0.223 

0.206 

0.683 

0.694 

0.011 

0.200 

0.188 

Average  %  Diff. 

0.007 

Average  %  Diff. 

0.016 

Table  11.  Dynamic  Optimization  Results  PTa  and  Pfta 


Static  Max  PTA 

Static  Final  PTFA 

Numeric 

Solution 

Simulated 

Results 

%  Diff. 

Numeric 

Solution 

Simulated 

Results 

%  Diff. 

Case  la:  c=100,  XF1=25,  PFTA  <1.0 

0.535 

0.533 

0.308 

0.010 

Case  lb:  c=100,  XFT  ~25,  PFTa  <  0.2 

0.482 

0.476 

0.179 

0.021 

Case  2a:  c=100, XFT  ~5,  PFTA  <  1.0 

0.153 

Case  2b:  c=100,  /.i  t  “5.  Pti.i  <  0.2 

0.153 

0.430 

0.428 

0.420 

0.398 

0.313 

0.301 

0.200 

0.179 

0.670 

0.685 

0.205 

0.007 

Case  4b:  c=50, 2FT  =5,  PFT4  <  0.2 

0.670 

0.685 

0.192 

0.008 

Average  %  Diff. 

0.008 

Average  %  Diff. 

0.012 

Table  12.  Static  Optimization  Results  PTa  and  Pfta 


The  results  presented  in  the  figures  and  tables  above  provide  strong  evidence  that 
MultiUAV  is  able  to  incorporate  time  varying  control  inputs  to  conduct  dynamic 
optimization  studies  of  wide  area  search  and  engagement  research  applications.  The 
performance  measures  produced  by  Rosario’s  numeric  optimization  algorithm, 
designated  as  the  Numeric  Solution  in  the  Tables  1 1  and  12,  are  within  a  99%  confidence 
interval  of  the  simulation  results. 
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As  expected,  the  above  results  also  show  that  the  simulations  implementing  time 
varying  PTr  schedules  consistently  produced  higher  final  values  of  Pta  than  the 
simulations  using  static  Ptr  values.  In  addition,  the  results  show  that  the  unconstrained 
simulations,  where  PFTa<1-0,  produced  higher  final  values  of  Pta  and  Pfta  than  the 
constrained  simulations  with  the  exception  of  Case  2.  In  the  constrained  simulation  of 
Case  2,  shown  by  Figure  19,  the  final  value  for  Pfta  is  significantly  lower  than  the  max 
allowed  value  of  0.2.  The  dynamic  optimization  results  from  Rosario’s  algorithm  show 
this  as  well  in  Tables  1 1  and  12.  This  implies  that  the  constraint  on  Pfta  is  inactive  and 
therefore  has  no  impact  on  the  resulting  schedule  for  Ptr.  This  allowed  for  identical 
results  for  the  constrained  and  unconstrained  simulations. 

Summary  of  Results  and  Analysis 

The  results  presented  in  this  chapter  have  demonstrated  the  ability  of  MultiUAV 
to  simulate  the  performance  of  an  ATR  process  with  a  multiple  warhead  capable  UCAV 
for  use  in  wide  area  search  and  destroy  research  applications  (Research  Objective  1). 
This  was  accomplished  through  the  use  of  several  hypothetical  trade  studies  performed 
using  data  generated  by  various  MultiUAV  simulations  and  comparing  this  data  to 
analytic  theory.  The  data  presented  in  this  chapter  also  demonstrated  the  ability  of 
MultiUAV  to  simulate  time  varying  control  parameters  to  dynamically  optimize  ATR 
performance  for  wide  area  search  and  engagement  research  applications  (Research 
Objective  2).  Chapter  5  will  present  the  final  conclusions  and  recommendations  of  this 
work. 
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V.  Conclusions  and  Recommendations 


Conclusions 

This  work  presented  a  method  to  validate  the  ability  of  a  simulation  environment 
such  as  MultiUAV  to  realistically  model  the  perfonnance  of  an  ATR  process  to 
discriminate  between  targets  and  false  targets  in  wide  area  search  and  engagement 
applications.  This  was  accomplished  by  comparing  results  generated  from  multiple 
simulations  of  academically  contrived  wide  area  search  and  engagement  scenarios  to 
closed  fonn  analytic  expressions  derived  for  the  same  scenarios.  The  ability  of 
MultiUAV  to  simulate  the  perfonnance  of  an  ATR  process  of  a  multiple  warhead  capable 
UCAV  and  the  ability  of  MultiUAV  to  incorporate  time  varying  parameters  to 
dynamically  optimize  ATR  performance  were  examined.  Also  presented  in  this  research 
was  a  limited  analysis  on  how  to  utilize  these  types  of  simulations  to  make  design 
decisions  in  the  development  of  wide  area  search  and  engagement  UCAV  systems  as  well 
as  determine  asset  allocations  (i.e.  UCAV  quantity  and/or  warhead  payload)  for  specific 
operational  missions. 

Contributions 

The  primary  goal  of  this  research  was  to  develop  a  methodology  to  validate  the 
baseline  performance  of  a  simulation  environment  for  wide  area  search  and  engagement 
research  applications.  Extensive  modification  to  the  subroutines  of  MultiUAV  was 
required  to  fulfill  this  objective.  An  additional  contribution  of  this  work  was  the  drawing 
together  of  the  key  algorithms  required  to  simulate  a  realistic  ATR  process.  These 
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algorithms  alone  can  be  the  basis  for  new  simulation  environments  and  future  research 
applications. 

Limitations  of  Research 

The  simulations  demonstrated  here  were  limited  to  very  simplistic  design 
parameters  intended  to  exactly  duplicate  academic  scenarios.  This  work  does  not  suggest 
that  MultiUAV  is  “off  the  shelf’  ready  to  model  an  ATR  process  as  presented  for  a 
cooperative  group  of  UCAVs.  Only  a  single  UCAV  was  simulated  in  this  work  which 
required  extensive  modifications  to  MultiUAV  to  produce  meaningful  results.  Additional 
work  is  required  to  model  the  ATR  process  of  multiple  UCAV’s  to  cooperatively 
classifying  encountered  targets. 

Recommendations  for  Future  Research 

1 .  Extend  this  work  with  MultiUAV  to  include  the  other  academic  scenarios  with 
a  multiple  warhead  capable  UCAV.  Continue  modifications  of  MultiUAV  to  allow  for 
multiple  UCAVs  to  cooperatively  classify  and  engage  encountered  targets  using  a 
modified  form  of  the  confusion  matrix  ATR  process  presented  here.  In  addition,  consider 
simulation  of  operational  scenarios  that  do  not  necessarily  lend  themselves  to  easy 
analytic  evaluation. 

2.  Pursue  development  of  an  alternative  simulation  environment  to  MultiUAV 
specifically  for  AFIT  research  applications.  The  modifications  made  to  MultiUAV  in 
this  line  of  research  represent  the  core  simulation  functions  required  to  model  wide  area 
search  and  engagement  vehicles.  MultiUAV  in  its  current  form  employs  high  fidelity 
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vehicle  dynamic  models  and  complicated  communication  subroutines  that  are 
unnecessary  for  some  research  applications.  A  new  simulation  could  be  developed  to 
perform  the  same  simulations  as  MultiUAV  with  a  simple  point  mass  vehicle  models  and 
stochastic  subroutines  to  model  ATR,  weapon  employment,  and  other  simulated 
functions. 

3.  Use  MultiUAV  or  an  alternative  simulation  environment  to  support  the  parallel 
development  of  a  small  scale  experimental  wide  area  search  and  engagement  vehicle. 

Summary 

The  use  of  computer  simulation  in  the  development  of  cooperatively  controlled 
unmanned  combat  aerial  vehicles  (UCAV)  for  autonomous  wide  area  search  and 
engagement  applications  was  addressed.  The  objective  of  this  research  was  to 
demonstrate  a  method  to  validate  a  simulation’s  ATR  model  for  future  use  in  the 
evaluation  of  cooperative  control  schemes.  This  was  accomplished  by  comparing  the 
results  from  multiple  simulations  of  academically  contrived  wide  area  search  and 
engagement  scenarios  to  closed  form  analytic  solutions  derived  for  the  same  scenarios. 
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Appendix  A:  MultiUAV  Target  Discovery  Time  Subroutines 


ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo 


Subroutine  TargetDiscoveryTimes . m 


ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo 


function  [TargetDiscoveryTimes]  =  TargetDiscoveryTimes (SearchTime, .. . 

Targets, FalseTargets) 


%  This  function  is  called  by  TargetSetup .m 
"6 

%  Inputs: 

%  SearchTime  -  Total  simulation  run  time 
%  Targets  -  Expected  number  of  target  encounters 
%  FalseTargets  -  Expected  number  of  false  target  encounters 


%  Outputs: 

%  TargetDiscoveryTimes (:, 1 ) 


%  TargetDiscoveryTimes (:, 2 ) 
"6 


Target  discovery  time  array  used  to 
determine  spatial  positions  of  targets 
based  on  search  agent  location  at 
the  discovery  time. 

Links  target  type  (i.e.  target  or  false 
target)  to  adjacent  discovery  time. 


%  AFIT/ENY 

%  December  2006  -  Created  and  Debugged  -  Marlin 


if  nargin==0, 

SearchTime=l ;  %  INPUT  -  total  search  time 
Targets=l;  %  INPUT  -  expected  real  targets 

FalseTargets=2 0 ;  %  INPUT  -  expected  false  targets 

end 

%  call  UniformDistribution 

[ Uni formDis cover yTimes ] =Unif ormDistr ibution (Targets ) 

%  associated  target  type 

Uni formDis cover yTimes ( : , 2 ) =ones ( 1 , length (Uni formDis cover yTimes ) ) ; 

%  call  PoissonDistribution 

[Poisson Discover yTimes ] =PoissonDistr ibution (FalseTargets ) 

%  associated  target  type 

PoissonDi sco very Times ( : , 2 ) =ones ( 1 , length ( Poiss on Dis cover yT ime s ) )  *2  ; 

%  sort  TargetDiscoveryTimes  in  sequential  order 
TargetDiscoveryTimes=sortrows ( [UniformDiscoveryTimes, 

PoissonDiscoveryTimes] , 1 ) ; 

%  scale  TargetDiscoveryTimes  to  SearchTime 

TargetDiscoveryTimes ( : , 1 ) =Tar get Dis cover yTimes ( : , 1 ) * SearchTime ; 
return 

'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O'O 

%  Subroutine  PoissonDiscoveryTimes .m 

ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo 

function  [PoissonDiscoveryTimes] =PoissonDistribution (ExpectedTargets ) 
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%  This  function  is  called  by  TargetDiscoveryTimes . m 
%  Inputs: 

%  ExpectedTargets  -  Number  of  expected  target  encounters 
%  of  any  target  type 


%  Outputs: 

%  PoissonDiscoveryTimes  -  Target  discovery  times  as  determined 
%  through  a  modeled  Poisson  process 

%  with  characterized  by  ExpectedTargets. 

%  Note:  Requires  Matalb  RANDOM  toolbox.  If  RANDOM  toolbox 
%  not  available  use  the  following  line  of  code: 

%  TimeStep (i) =-log (rand) /Lambda; 

%  AFIT/ENY 

%  December  2006  -  Created  and  Debugged  -  Marlin 


Lambda=ExpectedTargets ;  %  INPUT  -  Poisson  parameter 

DiscoveryTimes (1 ) =random (' Exponential 1/Lambda) ;  %  first  tgt  arrival  time 
i=l ; stop=0 ; 

while  (DiscoveryTimes (i) <1 )  &&  (stop==0), 

TimeStep=random (' Exponential 1/Lambda) ;  %  next  arrival  time 

if  TimeStep  <  1-DiscoveryTimes ( i ) ,  %  tgt  arrival  times  cannot  exceed  1 

DiscoveryTimes (i+1 ) =DiscoveryTimes (i ) +TimeStep;  %  build  arrivals 
i=i+l ; 

else  %  stop  if  next  arrival  would  exceed  1 
stop=l ; 

end 

end 

PoissonDiscoveryTimes=DiscoveryTimes ' ;  %  OUTPUT  to  TargetDiscoveryTimes 

return 


ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo 


%  Unif ormDiscoveryTimes . m 


ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo 


function  [Unif ormDiscoveryTimes ] =UniformDistribution (NumberTargets ) 

%  This  function  is  called  by  TargetDiscoveryTimes . m 

'O 

%  Inputs: 

%  ExpectedTargets  -  Number  of  target  encounters  specified 
%  Outputs : 

%  UniformDiscoveryTimes  -  Target  discovery  times  as  determined 
%  through  a  modeled  uniform  distribution. 


%  AFIT/ENY 

%  December  2006  -  Created  and  Debugged  -  Marlin 
"6 


Unif ormDiscoveryTimes=rand (NumberTargets , 1 ) ;  %  OUTPUT  to  TargetDiscoveryTimes 


ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo 


return 
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Appendix  B:  MultiUAV  Classify  Targets  Subroutines 


ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo 


Subroutine  Classif yTargets .m 


ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo 


function  Classification  =  ClassifyTarget (TrueTargetType, Pid) 
%  This  function  is  called  by  ATRFunctions . m 


%  Classification  -  Determines  what  the  vehicle  classifies  the  target  as 
%  based  on  the  vehicle's  probability  of  identification 

%  (Pid)  and  a  random  draw.  This  is  not  necessarily  what 

%  the  target  truly  is. 

"6 

%  Inputs: 

%  TrueTargetType  -  True  type  of  the  target 

%  Pid  -  Confusion  matrix 


Encountered  Object 


"6 

1 

2 

3 

Pid  = 

[0.000 

0.000 

0.000 

■"6 

0.000 

0.000 

0.000 

"o 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

15 

0.000 

0.000 

0.000 

"6 

"6 

Outputs : 

-o 

Classification 

-  What 

the  vehicle 

4  5 


0.000  0.000; . . 

%  1 

0.000  0.000;  .  . 

%  2 

0.000  0.000; . . 

%  3  Declared  Object 

0.000  0.000; . . 

%  4 

0.000  0.000]  ; 

%  5 

classified  the 

target  as 

%  AFIT/ENY 

%  September  2001  -  Created  and  Debugged  -  Dunkel 


RandomNumber  =  rand; 
j  =  TrueTargetType; 

Bounds  =  [Pid(l,j);  ... 

Pid (1, j ) +Pid (2 , j )  ;  ... 

Pid (1, j ) +Pid (2 , j ) +Pid (3,  j  )  ;  ... 

Pid ( 1 , j ) +Pid (2, j ) +Pid (3,  j ) +Pid (4,  j  )  ;  ... 

Pid ( 1 , j ) +Pid (2, j ) +Pid (3, j ) +Pid (4, j ) +Pid(5,  j ) ] ; 

%  The  variable  Bounds  is  used  to  determine  the 
%  relation  between  the  random  number  and  the 
%  confusion  matrix.  The  elements  of  bounds  are 
%  simply  the  progressive  summation  of  the  elements 
%  in  a  given  column  of  the  confusion  matrix  (Pid) . 


if  RandomNumber  <= 
Classification  = 
elseif  RandomNumber 
Classification  = 
elseif  RandomNumber 
Classification  = 
elseif  RandomNumber 
Classification  = 
elseif  RandomNumber 
Classification  = 


Bounds ( 1 ) 

1;  %  Target 

>  Bounds (1 ) 

2;  %  Target 

>  Bounds (2 ) 

3;  %  Target 

>  Bounds (3) 

4;  %  Target 

>  Bounds (4 ) 

5;  %  Target 


State  =  type (j ) -classified-type (1 ) 
&  RandomNumber  <=  Bounds (2) 

State  =  type (j ) -classified-type (2) 
&  RandomNumber  <=  Bounds (3) 

State  =  type (j ) -classified-type (3) 
&  RandomNumber  <=  Bounds (4) 

State  =  type (j ) -classified-type (4) 
&  RandomNumber  <=  Bounds (5) 

State  =  type (j ) -classified-type (5) 


end 


return; 
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